Adding nodes to the cluster using the CLI Scenario: typical usage for the command-line interface Displaying node properties using the CLI

This task provides step-by step instructions you will need to perform to add nodes to the cluster

Prerequisites

Before adding a node to the cluster check to see if any of the following conditions are true:

Attention: Attention: If any of these conditions are true, then you must perform the following special procedures. Failure to perform the special procedure is likely to result in the corruption of all data managed by the cluster.
Special procedures when adding a node to a cluster

If any of the previous conditions are true, then the following special procedures apply. These special procedures apply when you use either the svctask addnode command or the SAN Volume Controller Console. When a node is added to a cluster then either:

Hypothetical scenarios where the special procedures may apply.

The following are two hypothetical scenarios where the special procedures may apply:

Background

Applications on host systems direct I/O operations to file systems or logical volumes which are mapped by the operating system to vpaths which are pseudo disk objects supported by the SDD driver. See the IBM TotalStorage Subsystem Device Driver: User's Guide.

The SDD driver maintains an association between a vpath and a SAN Volume Controller VDisk. This association uses an identifier (UID) which is unique to the VDisk and is never re-used. This allows the SDD driver to unambiguously associate vpaths with VDisks.

The SDD device driver operates within a protocol stack which also contains Disk and Fibre Channel device drivers which allow it to communicate with the SAN Volume Controller using the SCSI protocol over Fibre Channel as defined by the ANSI FCS standard. The addressing scheme provided by these SCSI and Fibre channel device drivers uses a combination of a SCSI Logical unit number (LUN) and the World Wide Name for the Fibre Channel Node and Ports.

In the event of errors occurring, error recovery procedures (ERPs) operate at various tiers in the protocol stack. Some of these ERPs cause I/O to be redriven using the same WWN and LUN numbers which were previously used.

The SDD device driver does not check the association of the VDisk with the VPath on every I/O that it performs.

Data Corruption Scenario

Consider a four-node SAN Volume Controller configuration.

The nodes, Node1 and Node2, are in I/O group 0 which supports the VDisk, VDisk0.

The nodes, Node3 and Node4, are in I/O group 1 which supports the VDisk, VDisk1.

Assume that VDisk 0 is mapped to a host as LUN 0. This will be LUN 0 associated with the ports in Node1 and Node2. We might represent this as N1/0 and N2/0 respectively. Assume also that VDisk1 is also mapped to the host as LUN 0. Thus N3/0 and N4/0 are mapped to VDisk1.

Now assume that nodes, Node2 and Node4, are removed from the cluster.

If Node2 is added back into the cluster into I/O Group 1 a data corruption could occur because:

Context

Assume that the cluster has been created, that initial configuration has been performed using the SAN Volume Controller Console and that the necessary setup has been performed to use the Command Line Interface (CLI).

The following examples are all based on our hypothetical scenario of setting up a four node cluster. The first node has already been used to create a cluster and therefore there are a further three nodes to add to the cluster.

Prerequisites
Steps

Perform the following steps to add nodes to the cluster:

  1. Open a command prompt window.
  2. Type the svcinfo lsnode command to list the nodes that are currently part of the cluster.
    Example

    svcinfo lsnode -delim :
    
     id:name:UPS_serial_number:WWNN:status:IO_group_id:
     IO_group_name:config_node:UPS_unique_id
     1:node1:10L3ASH:500507680100002C:online:0:io_grp0:yes:202378101C0D18D8
    The cluster has only just been created so there is only one node in the cluster.
  3. Type the svcinfo lsnodecandidate command to list nodes that are not assigned to a cluster.
    Example

    svcinfo lsnodecandidate -delim :
    
     id:panel_name:UPS_serial_number:UPS_unique_id
     5005076801000001:000341:10L3ASH:202378101C0D18D8 
     5005076801000009:000237:10L3ANF:202378101C0D1796 
     50050768010000F4:001245:10L3ANF:202378101C0D1796 
    There are a total of four nodes, one of which has been used to create a cluster. Therefore there are three candidate nodes which can be added to the cluster.
  4. Attention: Attention: If you are re-adding a node to the SAN, ensure that you are adding the node to the same I/O group that it was removed from. Failure to do this can result in data corruption. Use the information that was recorded when the node was originally added to the cluster. If you do not have access to this information, call IBM Service to add the node back into the cluster without corrupting the data.

    This can avoid a possible data corruption exposure if the node must be removed from and re-added to the cluster.

    Note:Note: This warning also is displayed on the SAN Volume Controller Console panel when adding the node.
    Type the svctask addnode command to add a node to the cluster. Use the output from the previous commands to choose which I/O group to add the node to and to make sure that when adding a second node to a I/O group that it is attached to a different UPS.
    Notes:
    Notes:
    1. When adding a node to a cluster you can specify a name for the node. You can also change the name of nodes that are already part of a cluster using the svctask chnode command.
    2. When adding a node to a cluster the node can be identified by using the front panel name which is also printed on a label on the front of the SAN Volume Controller or by using the world wide node name of that node.
    Example

    Add a second node to the first I/O group. Note from the output from step 1 that the node that is already in I/O group 0 is attached to a UPS with the serial number 10L3ASH. Each node in an I/O group must be attached to a different UPS and therefore only the nodes with front panel IDs 000237 and 001245 are suitable candidates.

    svctask addnode -panelname 000237 -iogrp io_grp0 -name group1node2

    This command will add the node, identified by the front panel name 000237 to the cluster. The node will be added to I/O group, io_grp0, and called group1node2.

    Next add two nodes to the second I/O group. Check the output from step 3 to make sure that each node is attached to a different UPS.

    svctask addnode -wwnodename 5005076801000001 -iogrp io_grp1 -name group2node1
    svctask addnode -wwnodename 50050768010000F4 -iogrp io_grp1 -name group2node2

    These commands will add the nodes, identified by the WWNN 5005076801000001 and the WWNN 50050768010000F4 to the cluster. The nodes will be added to I/O group, io_grp1 and called group2node1 and group2node2.

    Finally change the name of the first node from the default name node1 so that it conforms with your naming convention.

    svctask chnode -name group1node1 node1
  5. Verify the final configuration using the svcinfo lsnode command.
    Example

    In our hypothetical scenario, the command to list the nodes is:

    svcinfo lsnode -delim :
    
     id:name:UPS_serial_number:WWNN:status:IO_group_id:
     IO_group_name:config_node:UPS_unique_id
     1:group1node1:10L3ASH:500507680100002C:online:0:io_grp0:yes:202378101C0D18D8
     2:group1node2:10L3ANF:5005076801000009:online:0:io_grp0:no:202378101C0D1796
     3:group2node1:10L3ASH:5005076801000001:online:1:io_grp1:no:202378101C0D18D8
     4:group2node2:10L3ANF:50050768010000F4:online:1:io_grp1:no:202378101C0D1796
    
    Note:Note: If this command is issued quickly after adding nodes to the cluster the status of the nodes may be adding rather than online indicating that the process of adding the nodes to the cluster is still in progress. You do not however have to wait for all the nodes to become online before continuing with the configuration process.

    Remember: Record the following information:

    • Node serial number
    • WWNN
    • All WWPNs
    • I/O group that contains the node

    This can avoid a possible data corruption exposure if the node must be removed from and re-added to the cluster.

Result

You have now added four nodes to one cluster. The nodes are split into two I/O groups.

Related topics Scenario: typical usage for the command-line interface Displaying node properties using the CLI

(C) Copyright International Business Machines Corporation 2003, 2004