Performing the node rescue Maintaining Recovering from offline VDisks after a node or an I/O group failed

If it is necessary to replace the hard disk drive or the software on the hard disk drive has become corrupted, you can reinstall the software on the SAN Volume Controller by using the node rescue procedure.

Context

To provide an alternate boot device, a minimal operating system is also available in nonvolatile memory on the service controller. If it is necessary to replace the hard disk drive or the software on the hard disk drive has become corrupted, the SAN Volume Controller cannot boot and the Hardware Boot indicator remains on the front panel display or the boot operation hangs.

If this occurs, you can reinstall the software on the SAN Volume Controller by using the node rescue procedure. Node rescue works by booting the operating system from the service controller and running a program that will copy all the node software from any other SAN Volume Controller that can be found on the fibre-channel fabric. The following procedure tells you how to run the node rescue procedure.

Steps

Perform the following steps to complete the node rescue:

  1. Ensure that the fibre-channel cables are connected.
  2. Ensure that at least one other SAN Volume Controller node is connected to the fibre-channel fabric.
  3. Turn off the SAN Volume Controller.
  4. Press and hold the left and right buttons on the front panel.
  5. Press the power button.
  6. Continue to hold the left and right buttons until the Node Rescue Request symbol is displayed on the front panel.
Figure 166. Node-rescue-request display
Node-rescue-request display
Result

The node rescue request symbol displays on the front panel display until the SAN Volume Controller starts to boot from the service controller. If the node rescue request symbol displays for more than two minutes, check the connection between the service controller and the system board. The service display shows the progress or failure of the node rescue operation.

Note:Note: If the recovered node was part of a cluster, the node will now be offline. Delete the offline node from the cluster and then add the node back into the cluster. If node recovery was used to recover a node that failed during a software upgrade process, the automatic software downgrade process will start but may not continue until the failed node is deleted from the cluster. After the failed node is deleted, it is not possible to add the node back into the cluster until the downgrade process has completed. This may take up to two hours.
Post-processing requirements

If the cables are correctly located and the node rescue request symbol still displays, replace the FRUs in the following sequence:

  1. System board assembly
  2. Service controller
Related topics Maintaining Recovering from offline VDisks after a node or an I/O group failed

(C) Copyright International Business Machines Corporation 2003, 2004