Description
This command removes a node from the cluster. This makes the node a candidate
to be added back into this cluster or into another cluster. After the node
is deleted, the other node in the I/O group destages the contents of its cache
and goes into write-through mode until another node is added back into the
I/O group.
Prerequisities:
Before you issue the rmnode command, perform the
following tasks and read the following Attention notices to avoid losing access
to data:
- Determine which virtual disks (VDisks) are still assigned to this I/O
group by issuing the following command. The command requests a filtered view
of the VDisks, where the filter attribute is the I/O group.
svcinfo lsvdisk -filtervalue IO_group_name=<name>
where <name> is the name of the I/O group in question.
 | Notes:
- If this is the last node in an I/O group and if there are virtual disks
still assigned to the I/O group, you will not be able
to delete the node from the cluster.
- Any VDisks that are assigned to the I/O group that this node belongs to,
will be assigned to the other node in the I/O group; that is, the preferred
node will be changed. You cannot change this setting back.
|
- Determine the hosts that the VDisks are mapped to by issuing the svcinfo lsvdiskhostmap command.
- Determine if any of the VDisks assigned to this I/O group contain data
that you need to maintain access to:
- If you do not want to maintain access to these
VDisks, go to step 5.
- If you do want to maintain access to some or all
of the VDisks, back up the data or migrate the data to a different (online)
I/O group.
- Determine if you need to turn the power off to the
node:
-
Update the SDD configuration for each virtual
path (vpath) that is presented by the VDisks that you intend to remove. Updating
the SDD configuration removes the vpaths from the VDisks. Failure to update
the configuration can result in data corruption. See the IBM TotalStorage Subsystem Device Driver: User's Guide for
details about how to dynamically reconfigure SDD for the given host operating
system.
- Quiesce all I/O operations that are destined for the node you are deleting.
Failure to quiesce the operations could result in failed I/O operations being
reported to your host operating systems.
 | Attention: Removing the last node in the cluster destroys the cluster. Before
you delete the last node in the cluster, ensure that you want to destroy the
cluster. |
 | Attention: If you are deleting a single node and the other node in the I/O
group is online, the cache on the partner node will go into write-through
mode, causing the data to be exposed to a single point of failure if the partner
node fails. |
 | Notes:
- If the node you are removing is the configuration node, it might take
a minute or more before you can perform the delete node request. You must
wait for the configuration node failover to occur.
- If the node you are removing is the last node in the cluster, the SAN Volume Controller Console
might seem to hang for up to 3 minutes because you have removed the last access
point to the cluster.
|
Deleting a node from a cluster:
 | Notes:
- If this is the last node in the I/O group or the last node in the cluster,
you will be asked to force the deletion.
- If this is the last node in the cluster or if it is currently assigned
as the configuration node, all connections to the cluster will be lost. The
user interface and any open CLI sessions will halt. A time-out might occur
if a command cannot be completed before the node is deleted.
|
Issue the svctask rmnode command to delete
a node from the cluster. You can enter this command any time after a cluster
has been created.
Adding a node back into the
cluster:
If you turn the power back on to the node that has been removed while it
is still connected to the same fabric or zone, the following actions occur:
- The node attempts to join the cluster again.
- The cluster signals the node to remove itself from the cluster.
- The node becomes a candidate for addition to this cluster or another cluster.
If you intend to add this node back into the cluster, ensure that you
add it back to the same I/O group from which you are deleting it. Otherwise,
data corruption might occur.
Before you add a node back into the cluster, you need to know the following
information, which should have been recorded when the node was originally
added to the cluster.
- Node serial number
- WWNN
- all WWPNs
- I/O group that contains the node
If you do not have access to this information, call IBM Service to add
the node back into the cluster without corrupting the data.
Replacing a faulty node:
You can replace a failed node with a"spare"or
replacement node. This would be necessary if the SAN Volume Controller fails.
The San Volume Controller will continue to operate with degraded performance
until the failed node is repaired. To enhance availability, you might choose
to replace the failed node with a "spare", then repair the node offline.
However, various procedures must be followed and precautions taken, to replace
a failed node without interrupting I/O and without risk to data integrity
when the repaired node is reconnected to the SAN fabric. The procedure involves
changing the World Wide Node Name (WWNN) of the SAN Volume Controller. This
procedure must be followed with care to avoid duplicate WWNNs which are illegal
and can cause data corruption.
Prerequisities:
Before replacing the failed node you must:
- Have SVC software version 1.1.1 or higher installed on the SAN Volume
Controller cluster and spare node.
- Know the cluster name that contains the failing node.
- Confirm that a spare node is in the same rack as the SAN Volume Controller
cluster containing the failed node.
- Make a record of the last five characters of the original WWNN. This identification
is needed if you decide in the future to designate the spare node as a normal
node that can be assigned to any cluster. Use the svcinfo_lsnode command to
see the WWNN.
Additional Information
When a node is replace using this procedure:
- The node Front Panel ID will change. This is the number printed on the
front of the node, used to select the node that is to be added to a cluster.
- The Node Name might change. If you permit the SAN Volume Controller Application
to assign default names when adding nodes to the cluster it will create a
new name each time a node is added. If you choose to assign your own names
then you will need to type in the node name that you wish to use. If you are
using scripts to perform management tasks on the cluster and those scripts
use the Node Name then by assigning the original name to a replacement node
you will avoid the need to make changes to the scripts following service activity
on the cluster.
- The Node ID will change. A new Node ID is assigned each time a node is
added to a cluster. The Node ID or the Node Name can be used when performing
management tasks on the cluster but if scripts are being used to perform those
tasks it is recommended that the Node Name is used in preference to the Node
ID since the Node Name will remain unchanged following service activity on
the cluster.
- The World Wide Node Name (WWNN) will not change. The WWNN is used to uniquely
identify the node and the fiber channel ports. The node replacement procedure
changes the WWNN of the spare node to match that of the failed node. The node
replacement procedures must be followed exactly to avoid any duplication of
WWNNs.
- The World Wide Port Name (WWPN) of each fiber channel port will not change.
The WWPNs are derived from the WWNN that is written to the replacement node
as part of this procedure.
Perform the following steps to replace a node:
- Use the command svcinfo lsnode to display the node
name. This command, when implemented, prints a detailed list report containing
information about all the nodes on a cluster. The failed node will be offline.
Note the nodes' name.
- Again use svcinfo lsnode to display the I/O Group
name. Note the name of the group.
- Use the command svcinfo lsnodevpd to display the
front panel id. Note the id number.
- Again use the command svcinfo lsnodevpd to record
the UPS serial number. Note this number.
- Use the Front Panel ID to locate the failed node. Disconnect all four
fiber channel cables from the node. Important: The cables
must not be reconnected until the node is repaired and the node number has
been changed to the default spare node number.
- Connect the power/signal cable from the spare node to the UPS with the
serial number noted in step 1. The signal cable can be plugged into any vacant
position on the top row of serial connectors on the UPS. If no spare serial
connectors are available on the UPS, disconnect the cables from the failed
SAN Volume Controller. Power-on the spare node. Display the node status on
the service panel. (See "SAN Volume Controller menu options"in the
SAN Volume Controller Service Guide).
Perform these steps to change the WWNN of the
node and add the replacement node:
- With the node status displayed on the front panel, press and hold the
Down button; press and release the Select button; release the Down button.
The text "WWNN"is displayed on line-1 of the display. Line-2 of the
display contains the last five characters of the WWNN.
- With the WWNN displayed on the service panel, press and hold the Down
button, press and release the Select button, release the Down button. This
switches the display into edit mode.
- Change the displayed number to match the WWNN recorded in step 1. To edit
the displayed number use the Up and Down buttons to increase or decrease the
numbers displayed. Use the left and right buttons to move between fields.
When the five characters match the number recorded in step 1, press the select
button twice to accept the number.
- Connect the four fiber channel cables that were disconnected from the
failed node to the spare node.
- Delete the offline node. See "Deleting
a node from the cluster".
- Add the spare node into the cluster. See "Adding
a node back into the cluster".
- Use the Subsystem Device Drive (SSD) management tool on the host systems
to verify all paths are now online.
When the failed node is repaired do not connect
the fiber channel cables to it. Connecting the cables might cause data corruption.
Perform these steps after the failed node is repaired:
- Display the node status on the service panel.
- With the SVC status displayed on the front panel, press and hold the Down
button; press and release the Select button; release the Down button. The
text "WWNN"is displayed on line-1 of the display; line-2 of the display
contains the last five characters of the WWNN.
- With the WWNN displayed on the service panel, press and hold the Down
button, press and release the Select button, release the Down button. This
switches the display into edit mode.
- Change the displayed number to "00000". To edit the displayed
number use the Up and Down buttons to increase or decrease the numbers displayed.
Use the left and right buttons to move between fields. When the number is
set to "00000", press the select button twice to accept the number.
Never connect a SVC with a WWNN of"00000" to the cluster.
This error should not be marked as fixed in the error log until the failed
node has been repaired and returned to the customer. If this is not done,
the service technician will not be able to easily find the front panel ID
of the failed node.
This SVC can now be used as a spare node. If this SVC is no longer required
as a spare and is to be used for normal attachment to a cluster you must first
use the procedure described above to change the WWNN to the number saved when
a spare was being created. See "Prerequisites"above. Using any other
number might cause data corruption.