This task provides step-by step instructions you will need to perform to add nodes to the cluster
Before adding a node to the cluster check to see if any of the following conditions are true:
| Attention: If any of these conditions are true, then you must perform the following special procedures. Failure to perform the special procedure is likely to result in the corruption of all data managed by the cluster. |
If any of the previous conditions are true, then the following special procedures apply. These special procedures apply when you use either the svctask addnode command or the SAN Volume Controller Console. When a node is added to a cluster then either:
Note: The WWNN of the nodes in the cluster can be determined
using the command:
svcinfo lsnode |
The following are two hypothetical scenarios where the special procedures may apply:
Applications on host systems direct I/O operations to file systems or logical volumes which are mapped by the operating system to vpaths which are pseudo disk objects supported by the SDD driver. See the IBM TotalStorage Subsystem Device Driver: User's Guide.
The SDD driver maintains an association between a vpath and a SAN Volume Controller VDisk. This association uses an identifier (UID) which is unique to the VDisk and is never re-used. This allows the SDD driver to unambiguously associate vpaths with VDisks.
The SDD device driver operates within a protocol stack which also contains Disk and Fibre Channel device drivers which allow it to communicate with the SAN Volume Controller using the SCSI protocol over Fibre Channel as defined by the ANSI FCS standard. The addressing scheme provided by these SCSI and Fibre channel device drivers uses a combination of a SCSI Logical unit number (LUN) and the World Wide Name for the Fibre Channel Node and Ports.
In the event of errors occurring, error recovery procedures (ERPs) operate at various tiers in the protocol stack. Some of these ERPs cause I/O to be redriven using the same WWN and LUN numbers which were previously used.
The SDD device driver does not check the association of the VDisk with the VPath on every I/O that it performs.
Consider a four-node SAN Volume Controller configuration.
The nodes, Node1 and Node2, are in I/O group 0 which supports the VDisk, VDisk0.
The nodes, Node3 and Node4, are in I/O group 1 which supports the VDisk, VDisk1.
Assume that VDisk 0 is mapped to a host as LUN 0. This will be LUN 0 associated with the ports in Node1 and Node2. We might represent this as N1/0 and N2/0 respectively. Assume also that VDisk1 is also mapped to the host as LUN 0. Thus N3/0 and N4/0 are mapped to VDisk1.
Now assume that nodes, Node2 and Node4, are removed from the cluster.
If Node2 is added back into the cluster into I/O Group 1 a data corruption could occur because:
Assume that the cluster has been created, that initial configuration has been performed using the SAN Volume Controller Console and that the necessary setup has been performed to use the Command Line Interface (CLI).
The following examples are all based on our hypothetical scenario of setting up a four node cluster. The first node has already been used to create a cluster and therefore there are a further three nodes to add to the cluster.
Perform the following steps to add nodes to the cluster:
svcinfo lsnode -delim : id:name:UPS_serial_number:WWNN:status:IO_group_id: IO_group_name:config_node:UPS_unique_id 1:node1:10L3ASH:500507680100002C:online:0:io_grp0:yes:202378101C0D18D8The cluster has only just been created so there is only one node in the cluster.
svcinfo lsnodecandidate -delim : id:panel_name:UPS_serial_number:UPS_unique_id 5005076801000001:000341:10L3ASH:202378101C0D18D8 5005076801000009:000237:10L3ANF:202378101C0D1796 50050768010000F4:001245:10L3ANF:202378101C0D1796
| Attention: If you are re-adding a node to the SAN, ensure that you are
adding the node to the same I/O group that it was removed from. Failure to
do this can result in data corruption. Use the information that was recorded
when the node was originally added to the cluster. If you do not have access
to this information, call IBM Service to add the node back into the cluster
without corrupting the data.
This can avoid a possible data corruption exposure if the node must be removed from and re-added to the cluster.
|
Notes:
|
Add a second node to the first I/O group. Note from the output from step 1 that the node that is already in I/O group 0 is attached to a UPS with the serial number 10L3ASH. Each node in an I/O group must be attached to a different UPS and therefore only the nodes with front panel IDs 000237 and 001245 are suitable candidates.
svctask addnode -panelname 000237 -iogrp io_grp0 -name group1node2
This command will add the node, identified by the front panel name 000237 to the cluster. The node will be added to I/O group, io_grp0, and called group1node2.
Next add two nodes to the second I/O group. Check the output from step 3 to make sure that each node is attached to a different UPS.
svctask addnode -wwnodename 5005076801000001 -iogrp io_grp1 -name group2node1 svctask addnode -wwnodename 50050768010000F4 -iogrp io_grp1 -name group2node2
These commands will add the nodes, identified by the WWNN 5005076801000001 and the WWNN 50050768010000F4 to the cluster. The nodes will be added to I/O group, io_grp1 and called group2node1 and group2node2.
Finally change the name of the first node from the default name node1 so that it conforms with your naming convention.
svctask chnode -name group1node1 node1
In our hypothetical scenario, the command to list the nodes is:
svcinfo lsnode -delim : id:name:UPS_serial_number:WWNN:status:IO_group_id: IO_group_name:config_node:UPS_unique_id 1:group1node1:10L3ASH:500507680100002C:online:0:io_grp0:yes:202378101C0D18D8 2:group1node2:10L3ANF:5005076801000009:online:0:io_grp0:no:202378101C0D1796 3:group2node1:10L3ASH:5005076801000001:online:1:io_grp1:no:202378101C0D18D8 4:group2node2:10L3ANF:50050768010000F4:online:1:io_grp1:no:202378101C0D1796
| Note: If this command is issued quickly after adding nodes to the cluster the status of the nodes may be adding rather than online indicating that the process of adding the nodes to the cluster is still in progress. You do not however have to wait for all the nodes to become online before continuing with the configuration process. |
Remember: Record the following information:
This can avoid a possible data corruption exposure if the node must be removed from and re-added to the cluster.
You have now added four nodes to one cluster. The nodes are split into two I/O groups.