IBM XIV 2810-A14 逻辑维护指南

IBM XIV Storage System

Model 2810 - A14

Maintenance Guide

Version 0.2

July 23, 2008

About this Document 3

Notes and Warnings 4

Unless specifically instructed do not press or use any button on the system. In particular, do not use the buttons on the ATS to power down the system. This bypasses the battery backup and will cause loss of data if done on an operational system. Always use the GUI or XCLI to stop and power down the system when necessary. 4

Tools 5

Getting Started 6

Attaching Laptop 6

Check System State 6

Physical Inspection 6

System State 7

Visual Indicators 7

Component States 13

Gather Data 13

Correct Component 13

Maintenance Instructions 14

Disk Drive Assembly 14

Data Module & Interface Modules 15

Module Power Supply 15

UPS Battery Pair 15

Uninterruptible Power Supply 15

UPS Network Management Card 16

Ethernet Switch 16

Ethernet Switch RPS 16

Maintenance Module 16

ATS 16

Modem 16

Cables 16

Special Cases 17

Network Connections 17

Patch Panel Laptop Port 17

Patch Panel Management Port 17

Patch Panel Remote Support (VPN) Port 17

Patch Panel Data Connections 17

Apparent Multiple Failed Components 18

UPS 18

Modules 4, 5 or 6 19

Ethernet Switch 19

Ethernet Switch RPS 19

Internal Power Cables 20

About this Document

This document provides information for diagnosing problems and servicing the IBM XIV model 2810-A14 storage system. The scope of this document includes problem diagnosis and the interactions with the system control software which are required when performing service actions. The audience for this document is the IBM field support team.

This document is a companion to IBM XIV Storage System Installation and Service Guide for Service Support Representatives (SSR), which documents the procedures for physical service actions on the system.

The following specific topics are covered:

1. Notes and Warnings – things you need to be aware of before attempting any service action
2. Tools – the software tools you will need to have installed on your laptop in order to perform service actions
3. Getting started – things that must be done with the system before starting any service procedure
4. Attaching laptop – how to attach a laptop to the laptop port and start the service GUI
5. Check machine status – things to check every time
6. Maintenance Instructions – how the maintenance process is done, and details instructions for each component
7. Special Cases

This document does not cover performing a code upgrade on the XIV system; this will be covered in a later release or a separate document.

Notes and Warnings

Never power down the system unless specifically instructed by the service procedure. If the system is in a unexpected state or a service procedure fails, unless there is a safety concern, always leave the system powered and contact the next level of support. Powering down the system almost always makes it harder or impossible to understand the failure and to recover the system.

The machine is serviced only when in Full Redundancy state. The Getting Started section below describes how to check the system state. The system is self-healing and will automatically recover to Full Redundancy state if possible. For a module failure this can take up to six hours after the failure. If the system does not come back to Fully Redundancy state always escalate to the next level of support for an action plan before attempting to service the machine.

Once any part of the machine has been installed into a system it must never be moved to a different position in the same system or moved to a new system. Once a component is removed from the system it must be replaced with a FRU part. Certain components, including the disk drive assemblies, data modules, interfaces modules and ethernet switches can cause serious system malfunctions if moved to a different position in the same system or moved to a different system.

Unless specifically instructed in a service procedure, never use the buttons on the UPS or the module to turn off a UPS or module. Turning off power with the UPS power buttons, in particular, can lead to data loss.

If more than one part has failed and needs to be repaired, escalate to the next level of support for an action plan. The order of replacing parts may be important if there are multiple failures.

Cables have to plug into the correct ports for the system to function correctly. All cables are labeled with the correct port. On the ethernet switches the cables must be plugged into the exact port indicated on the cable.

Unless specifically instructed do not press or use any button on the system. In particular, do not use the buttons on the ATS to power down the system. This bypasses the battery backup and will cause loss of data if done on an operational system. Always use the GUI or XCLI to stop and power down the system when necessary.

Tools

Include instructions for installing the GUI – need to get updated name for the GUI and where to obtain it.

Install the simple DOS BAT files to gather basic system info. (?)

If the laptop will be used by someone performing more advanced debug, also install PuTTY and WinSCP.

Basic instructions for navigating the GUI?

Getting Started

Attaching Laptop

If the laptop being used to service the system has wireless networking we recommend disabling the wireless capability during the service procedure. This will reduce the chance of having network connectivity problems.

Attach the laptop to one of the two “Laptop” ports on the patch panel using the blue CAT5 cable supplied with the system. Normally the laptop will receive an IP address via DHCP.

In case the laptop does not have DHCP enabled for some reason these are the network parameters to use for manual network configuration:

IP Address: 14.10.201.1

Net Mask: 255.255.255.0

Gateway: none

<Add section on how to use ipconfig to check that you got a good connection and force a retry if you didn’t>

In most cases it doesn’t matter which of the two “Laptop” ports on the patch panel is used to connect to the system. If either Module 4 or Module 5 has failed or is going to be serviced, see the Special Cases section below for instructions on which port to use.

Check System State

Before any service procedure perform the following checks. The goal of these checks is to determine if the system is in the appropriate state to be ready for service and if there have been any other failures that have not yet been recognized with might need to the resolved first.

Physical Inspection

Quick check that everything seems to be in place, cables plugged tightly, and there are no safety concerns.

System State

If the system state is not Full Redundancy, understand why and get it to Full Redundancy before doing the repair. Escalate to next level of support before proceeding if the system is not in Full Redundancy state.

Visual Indicators

Detail the “normal operation” visual indicators. If not as expected (except for the part to be repaired) figure out why before doing the repair.

Disk Drive LEDs

Module LEDs

PowerAttentionIDModule Front

(Probably need to do better drawings with PP and insert)

Module Rear

There are also indicators on each DIMM slot, visible inside of the chassis, which are normally off. Any of these on indicates a memory error in the system.
Switch LEDs

UPS LEDs

ATS LEDs

Examine the two input power LEDs on the ATS; both should be solid green. If not the system may not have redundant power input. Input power is a customer responsibility, so if the system does not have redundant input the customer should be notified.

Examine the output power LEDs on the ATS to the three UPS units and the management module. These should be solid green. If they are not green check that power is being supplied to the appropriate component. If you have been sent to service one of the UPSs or the maintenance module and the corresponding LED is not green on the ATS, the problem is most likely at the ATS.

<Picture>
Events

Use the GUI to check events. If the events show failures not associated with the part to be repaired, understand why before proceeding.

Component States

Check the GUI to see if there are components (other than the component to be serviced) which are no OK (failed or ready).

We might want to provide instructions to do this from the command line as well, in case there is any problem with the GUI not reporting things. Should we provide instructions to run on the command line, or a BAT file to do this?

xcli -u <用户名> -p <密码> -m %ipaddr% component_list filter=notok

xcli -u <用户名> -p <密码> -m %ipaddr% module_list

xcli -u <用户名> -p <密码> -m %ipaddr% ups_list

xcli -u <用户名> -p <密码> -m %ipaddr% switch_list

xcli -u <用户名> -p <密码> -m %ipaddr% psu_list

Gather Data

Run the da_snap program to collect system status information before starting the repair action.

Correct Component

Double check that the visual indicator matches the component that is about to be repaired.

Maintenance Instructions

In general replacing a component has the following steps.

1. The component to be replaced will be in Failed state (RED on the GUI).
2. In some cases you may be instructed to Phase Out a component that needs to be replaced but has not yet failed. After the phase out complete the component will be in Failed state.
3. The component is physically replaced, using the instructions in the IBM XIV Storage System Installation and Service Guide for Service Support Representatives document.
4. The new component is Tested. While the component test is running the component will be in Initializing state.
5. When the component test completes successfully the component will be in Ready state (Yellow on the GUI).
6. The new component is Phased In. For disks or modules the system will be in Redistributing state while the phase in proceeds.
7. When Phase In completes the component will be in OK state (uncolored on the GUI).

Some components don’t need to be phased in – testing takes them to OK state. We need a table which components require phase in and which only need test.

If the system doesn’t follow these state transitions escalate to next level of support for instructions before attempting anything else.

When replacing disk drive assemblies or modules the service procedure is complete when the system goes to Redistributing state. The SSR does not wait for this to complete – in some cases it will require hours for module replacements.

When the repair action completes run the da_snap program again to collect the system information after the repair.

Disk Drive Assembly

Data Module & Interface Modules

Module Power Supply

The power supply will go to Failed state for loss of A/C input or a failure of the component. If A/C power is restored it will automatically go back to OK. If the component fails and is physically replaced, it must be tested to go to OK.

No part replacement:

Part is replaced:

UPS Battery Pair

Uninterruptible Power Supply

After replacement a UPS must be tested and phased in.

UPS Network Management Card

Since replacing the network management card requires powering down and physically removing the UPS assembly, it will require the same actions as when replacing a UPS to integrate the UPS back into the system.

Ethernet Switch

The switch requires special attention because it is necessary to attach only certain CAT5 connections, power up the new switch, test it and then attach the remaining cables.

Ethernet Switch RPS

A failed RPS will show up on the GUI as critical indicator on both ethernet switches, not as a failed RPS. The detail bubble for both ethernet switches will show DC power failure.

After physically replacing the RPS the ethernet switches will automatically go back to OK state – it is not required to test the RPS.

Maintenance Module

ATS

Modem

Cables

Special Cases

There are direct communication paths into certain modules from the patch panel or from certain modules to other system components. This is true for all of the interface modules. This may cause other failures to be reported, and needs to be accounted for in selecting which laptop port to use when servicing the machine.

Network Connections

Patch Panel Laptop Port

There are two laptop ports on the patch panel; the upper port connects to module 5 and the lower port to module 4. If module 5 has failed or is being serviced, used the module 4 port, and likewise if module 5 has failed or is being serviced use the module 4 port.

These are the secondary error conditions which can be observed as a result of an interface module failure.

Patch Panel Management Port

The system has 3 redundant connections for the customer to perform system management, which are connected through modules 4, 5 and 6. At install time the customer is strongly encourage to use all three connections, so losing one of these modules will not interfere with the customer’s ability to manage the system. If the customer only uses one path and the module for that connection goes down they will lose the ability to manage the system.

Patch Panel Remote Support (VPN) Port

There are two ports on the patch panel for a network connection for report support. These connect to modules 6 and 4. If only one of the connections is used and the module is served the remote support connection will be lost.

Patch Panel Data Connections

Clients using the system connect using Fibre Channel or iSCSI connections through the patch panel, and these connections are in turn to the six interface modules. Customers are strongly encourage to always employ multipathing for these connections. In the case that multipathing is not used, or there are problems in the customer environment on one path in the connection, a failed interface module will also show a host connection error.

Apparent Multiple Failed Components

UPS

When a UPS fails all of the components supplied by the UPS will report power failures.

These components will report power loss if UPS 1 fails.

These components will report power loss if UPS 2 fails.

These components will report power loss if UPS 2 fails.

Modules 4, 5 or 6

There are direct communication connections between modules 4, 5 and 6 and the uninterruptible power supplies:

If one of these modules fails the corresponding UPS will also be shown as failed. The UPS continues to provide power and the system has a redundant path to communicate with the UPS through a serial connection; the UPS is shown as failed because the primary communication path to it has failed.

After the module has been repaired the primary communication path will be reestablished and the UPS will return to OK state. THIS NEEDS TO BE CONFIRMED.

You can check that the redundant communication path to the UPS is operational using the ups_list XCLI command. If the UPS shows data for the Currently Functioning, Input Power On, Battery Charge Level, Last Self Test Date, and Last Self Test Result fields, the redundant communication path is working correctly.

Ethernet Switch

Ethernet Switch RPS

A failed RPS will be reported as loss of DC power to both ethernet switches. There is not error message directly reporting that the RPS itself has failed.

Internal Power Cables

A simultaneous report of input power loss to two data modules may mean that an internal power cord has failed or become disconnected at the UPS connection. The following table shows the correlation between the internal power cables and the modules which they supply. If two modules on the same row simultaneously lose power, check the cable.