rmnode

You can use the rmnode command to delete a node from the cluster. You can enter this command any time after a cluster has been created.

Syntax

Read syntax diagramSkip visual syntax diagram>>- svctask -- -- rmnode -- --+- node_name -+------------------><
                              '- node_id ---'

Parameters

node_name | node_id
Specifies the node to be deleted. The argument is either:
  • The node name, that is, the label that you assigned when you added the node to the cluster
  • The node ID that is assigned to the node (not the WWNN).

Description

This command removes a node from the cluster. This makes the node a candidate to be added back into this cluster or into another cluster. After the node is deleted, the other node in the I/O group destages the contents of its cache and goes into write-through mode until another node is added back into the I/O group.

Prerequisities:

Before you issue the rmnode command, perform the following tasks and read the following Attention notices to avoid losing access to data:

  1. Determine which virtual disks (VDisks) are still assigned to this I/O group by issuing the following command. The command requests a filtered view of the VDisks, where the filter attribute is the I/O group.
    svcinfo lsvdisk -filtervalue IO_group_name=<name>
    where <name> is the name of the I/O group in question.
    Notes:
    Notes:
    1. If this is the last node in an I/O group and if there are virtual disks still assigned to the I/O group, you will not be able to delete the node from the cluster.
    2. Any VDisks that are assigned to the I/O group that this node belongs to, will be assigned to the other node in the I/O group; that is, the preferred node will be changed. You cannot change this setting back.

  2. Determine the hosts that the VDisks are mapped to by issuing the svcinfo lsvdiskhostmap command.
  3. Determine if any of the VDisks assigned to this I/O group contain data that you need to maintain access to:
    • If you do not want to maintain access to these VDisks, go to step 5.
    • If you do want to maintain access to some or all of the VDisks, back up the data or migrate the data to a different (online) I/O group.
  4. Determine if you need to turn the power off to the node:
    • If this is the last node in the cluster, you do not need to turn the power off to the node. Go to step 5.
    • If this is not the last node in the cluster, turn the power off to the node that you intend to remove. This step ensures that the Subsystem Device Driver (SDD) does not rediscover the paths that are manually removed before you issue the delete node request.

      If you plan to add the node back into the cluster, see ***.

  5. Update the SDD configuration for each virtual path (vpath) that is presented by the VDisks that you intend to remove. Updating the SDD configuration removes the vpaths from the VDisks. Failure to update the configuration can result in data corruption. See the IBM TotalStorage Subsystem Device Driver: User's Guide for details about how to dynamically reconfigure SDD for the given host operating system.
  6. Quiesce all I/O operations that are destined for the node you are deleting. Failure to quiesce the operations could result in failed I/O operations being reported to your host operating systems.
Attention: Attention: Removing the last node in the cluster destroys the cluster. Before you delete the last node in the cluster, ensure that you want to destroy the cluster.
Attention: Attention: If you are deleting a single node and the other node in the I/O group is online, the cache on the partner node will go into write-through mode, causing the data to be exposed to a single point of failure if the partner node fails.
Notes:
Notes:
  1. If the node you are removing is the configuration node, it might take a minute or more before you can perform the delete node request. You must wait for the configuration node failover to occur.
  2. If the node you are removing is the last node in the cluster, the SAN Volume Controller Console might seem to hang for up to 3 minutes because you have removed the last access point to the cluster.

Deleting a node from a cluster:

Notes:
Notes:
  1. If this is the last node in the I/O group or the last node in the cluster, you will be asked to force the deletion.
  2. If this is the last node in the cluster or if it is currently assigned as the configuration node, all connections to the cluster will be lost. The user interface and any open CLI sessions will halt. A time-out might occur if a command cannot be completed before the node is deleted.

Issue the svctask rmnode command to delete a node from the cluster. You can enter this command any time after a cluster has been created.

Adding a node back into the cluster:

If you turn the power back on to the node that has been removed while it is still connected to the same fabric or zone, the following actions occur:

  1. The node attempts to join the cluster again.
  2. The cluster signals the node to remove itself from the cluster.
  3. The node becomes a candidate for addition to this cluster or another cluster.

If you intend to add this node back into the cluster, ensure that you add it back to the same I/O group from which you are deleting it. Otherwise, data corruption might occur.

Before you add a node back into the cluster, you need to know the following information, which should have been recorded when the node was originally added to the cluster.

If you do not have access to this information, call IBM Service to add the node back into the cluster without corrupting the data.

Replacing a faulty node:

You can replace a failed node with a"spare"or replacement node. This would be necessary if the SAN Volume Controller fails. The San Volume Controller will continue to operate with degraded performance until the failed node is repaired. To enhance availability, you might choose to replace the failed node with a "spare", then repair the node offline. However, various procedures must be followed and precautions taken, to replace a failed node without interrupting I/O and without risk to data integrity when the repaired node is reconnected to the SAN fabric. The procedure involves changing the World Wide Node Name (WWNN) of the SAN Volume Controller. This procedure must be followed with care to avoid duplicate WWNNs which are illegal and can cause data corruption.

Prerequisities:

Before replacing the failed node you must:

Additional Information

When a node is replace using this procedure:

Perform the following steps to replace a node:

  1. Use the command svcinfo lsnode to display the node name. This command, when implemented, prints a detailed list report containing information about all the nodes on a cluster. The failed node will be offline. Note the nodes' name.
  2. Again use svcinfo lsnode to display the I/O Group name. Note the name of the group.
  3. Use the command svcinfo lsnodevpd to display the front panel id. Note the id number.
  4. Again use the command svcinfo lsnodevpd to record the UPS serial number. Note this number.
  5. Use the Front Panel ID to locate the failed node. Disconnect all four fiber channel cables from the node. Important: The cables must not be reconnected until the node is repaired and the node number has been changed to the default spare node number.
  6. Connect the power/signal cable from the spare node to the UPS with the serial number noted in step 1. The signal cable can be plugged into any vacant position on the top row of serial connectors on the UPS. If no spare serial connectors are available on the UPS, disconnect the cables from the failed SAN Volume Controller. Power-on the spare node. Display the node status on the service panel. (See "SAN Volume Controller menu options"in the SAN Volume Controller Service Guide).

Perform these steps to change the WWNN of the node and add the replacement node:

  1. With the node status displayed on the front panel, press and hold the Down button; press and release the Select button; release the Down button. The text "WWNN"is displayed on line-1 of the display. Line-2 of the display contains the last five characters of the WWNN.
  2. With the WWNN displayed on the service panel, press and hold the Down button, press and release the Select button, release the Down button. This switches the display into edit mode.
  3. Change the displayed number to match the WWNN recorded in step 1. To edit the displayed number use the Up and Down buttons to increase or decrease the numbers displayed. Use the left and right buttons to move between fields. When the five characters match the number recorded in step 1, press the select button twice to accept the number.
  4. Connect the four fiber channel cables that were disconnected from the failed node to the spare node.
  5. Delete the offline node. See "Deleting a node from the cluster".
  6. Add the spare node into the cluster. See "Adding a node back into the cluster".
  7. Use the Subsystem Device Drive (SSD) management tool on the host systems to verify all paths are now online.

When the failed node is repaired do not connect the fiber channel cables to it. Connecting the cables might cause data corruption. Perform these steps after the failed node is repaired:

  1. Display the node status on the service panel.
  2. With the SVC status displayed on the front panel, press and hold the Down button; press and release the Select button; release the Down button. The text "WWNN"is displayed on line-1 of the display; line-2 of the display contains the last five characters of the WWNN.
  3. With the WWNN displayed on the service panel, press and hold the Down button, press and release the Select button, release the Down button. This switches the display into edit mode.
  4. Change the displayed number to "00000". To edit the displayed number use the Up and Down buttons to increase or decrease the numbers displayed. Use the left and right buttons to move between fields. When the number is set to "00000", press the select button twice to accept the number. Never connect a SVC with a WWNN of"00000" to the cluster.

This error should not be marked as fixed in the error log until the failed node has been repaired and returned to the customer. If this is not done, the service technician will not be able to easily find the front panel ID of the failed node.

This SVC can now be used as a spare node. If this SVC is no longer required as a spare and is to be used for normal attachment to a cluster you must first use the procedure described above to change the WWNN to the number saved when a spare was being created. See "Prerequisites"above. Using any other number might cause data corruption.

Possible failures

Examples

An invocation example

svctask rmnode 1