正文文本
DS8000 Backend Loop Briefing
Liu Qi (EC LSG)
2
Agenda
DS8000 backend loop general
DS8300 & DS8700 backend loop
Hardware
Loop connection
DA takeover
DS8800 & DS8870 backend loop
Hardware
Loop connection
Backend loop problem determination
issmap reading
Error loop debug
Backend loop general
DS8000 Backend loop, a switched Fiber Channel Arbitrated Loop (switched FCAL) ,consist of Device Adapter, FCIC, DDM, and cable.
Switched FCAL is controlled by the DA adapters and SES code (Storage Enclosure Services) on the FCIC cards.
DA adapter works in pair. Each DA adapter control 4 loops, 1R/1L/2R/2L, which are known as dual loop 1 and dual loop 2.
3
Backend loop general (Continue)
Each DDM has two ports, one for each FCIC card in the storage enclosure.
Each DDM are accessible by four paths.
4
DS8300 & DS8700 backend loop
Device Adapter (Arrowhead)
Four port
2Gbps FC each port
PCI-X bus
5
DS8300 & DS8700 backend loop
Storage Enclosure (Megapack)
Disk Technology
3.5” (LFF) Fibre Channel
Throughput
2Gbps FC interconnect backbone
2Gbps FC to disks
Density
Supports 16 disks per enclosure
3.5U of vertical rack space
Cabling
Passive copper interconnect
Modularity
Rack level power
Rack level cooling
6
Location
Rx-Syz
x represents to rack number;
y represents to position of rack. 1 is front, 2 is rear.
z represents to sequence of enclosure from up to down.
DA – FCIC cabling rule
Each DA control 4 loops, 1R/1L/2R/2L
From up to down, the four DA ports are 2R, 2L, 1R, 1L respectively.
Dual loop 2 consists of DA and megapack in front of Rack. Dual loop 1 consists of DA and megapack in back of Rack.
‘ L’ ports of DA connect to right hand FCIC of Megapack. ‘ R’ ports of DA connect to left hand FCIC of Megapack.
Ports of ‘even’ DA always connect to T1 port of FCIC. Ports of ‘odd’ DA always connect to T2 port of FCIC.
T3 and T4 port of FCIC connect to T1 and T2 port of downstream FCIC respectively in a cascading situation.
For example:
DA iss002 T2 port control 2L loop. It connects to right hand FCIC T1 port of a Megapack at front of the base rack.
7
DA – Megapack cabling in rack view
8
A simple diagram of DA – FCIC cabling
9
Sparing Rule
When the arrays are created on a DS8000, the microcode determines which array sites will contain spares. The first array sites on each DA pair that are assigned to arrays will contribute one or two spares (depending on the RAID option), until the DA pair has access to at least four spares, with two spares being placed on each loop.
A minimum of one spare is created for each array site assigned to an array until the following conditions are met:
There are a minimum of 4 spares per DA pair.
There are a minimum of 4 spares for the largest capacity array site on the DA pair.
There are a minimum of 2 spares of capacity and RPM greater than or equal to the fastest array site of any given capacity on the DA pair.
***Above Sparing Rule is identical on all models of DS8000 series. ***
10
DA takeover function
Definition:
operational LPAR taking ownership of a remote Device Adapter (DA) when local DA and remote LPAR both fail.
Expectations:
avoid access loss in the above dual-failure scenario
IO routed through the surviving LPAR and the remote DA(s)
Existence:
DS8000 microcode R4 and above (including DS8700 and DS8800).
11
Normal Operation
12
Dual LPAR
No fenced or quiesced DAs
DA takeover function
13
Single Failure - DA
Dual LPAR
Fenced or quiesced local DA
No access loss due to DA Failover.
DA takeover function
14
Single Failure - LPAR
Single LPAR (remote LPAR fenced or quiesced)
No fenced or quiesced local DAs
No access loss due to LPAR failover ERP (already existing ERP)
DA takeover function
15
Dual Failure - LPAR and DA
Single LPAR (remote LPAR fenced of quiesced)
At least one fenced local DA.
This is an access loss scenario in pre-R4.0 code.
No access loss due to DA Takeover ERP (new ERP in R4.0+ code)
DA takeover function
DA takeover function
Recovery
Repair local DA card first. Then repair remote CEC (the failed CEC)
****When DA takeover is triggered, it is a serious and complex issue. PFE MUST be engaged for recovery.****
Limitation (R4 and R5.0 only)
No recursive DA takeover on one adapter pair before returning back to Dual LPAR operation. Which means, during recovery of DA takeover situation, if local DA card failed again after its recovery while machine is still in single cluster mode, DA takeover will NOT happen again. Access loss hit!!
16
DS8800 & DS8870 backend loop
17
Device Adapter (Dorinda)
1.5Ghz dual core Freescale MPC8572 processor
1 GB data store
512MB processor memory
8Gbps port
DS8800 & DS8870 backend loop
18
Storage Enclosure (Gigapack)
Disk Technology
2.5” (SFF) SAS
Throughput
8Gbps FC interconnect backbone
6Gbps SAS to disks
Density
Supports 24 disks per enclosure
2U of vertical rack space
Cabling
Optical short wave multimode interconnect
Modularity
Integrated power
Integrated cooling
Location
Rx-Gyy:
x represents to rack number;
yy represents to sequence of enclosure from up to down.
DA – FCIC cabling rule
Each DA control 4 loops, 1R/1L/2R/2L
From up to down, the four DA ports are 1L, 1R, 2L, 2R respectively.
‘ L’ ports of DA connect to left hand FCIC of Gegapack. ‘ R’ ports of DA connect to right hand FCIC of Gegapack.
Ports of ‘even’ DA always connect to T1 port of FCIC. Ports of ‘odd’ DA always connect to T2 port of FCIC.
T3 and T4 port of FCIC connect to T1 and T2 port of downstream FCIC respectively in a cascading situation.
DS8800 DA - FCIC cabling rules fit human habits
19
A simple diagram of DA – FCIC cabling (DS8800 & DS8870)
20
Standard class cabling in rack view
21
DS8800 Business class cabling in rack view
22
DS8870 Business class cabling in rack view (reduced drive support from R6, upgrades to Enterprise class.)
23
144 drives
2 active processor cores
16GB or 32GB
No SSD support
No copy services support
No I/O priority manager support
Standard class cabling in rack view (DS8800 & DS8870)
24
DA card pair
DA port pair
Position in loop
2 a 2
Backend Loop Problem Determination
issmap
issmap provide a detailed diagnostic to the backend loop.
issmap is developed by Arrowhead firmware. So issmap output is most close to the physical fact.
Run issmap in LPAR, not in HMC.
issmap in button tool
25
issmap -l issX | -a [-b L/R][-c][-e][-f][-g][-h][-k][-m][-n][-p][-s][-t]
where: -l issX Specifies adapter to use
-a Display for all adapters (do not use with -l)
-b L/R Specifies the adapter port from which to view
the nodes
-c Display Code level
-e List with Invalid Transmission Word Counts
-f Display with AL_PA Fibre Channel Address
-g Display with location information (enclosure-slot
-h Display with logical numbers/names
(eg. hdiskX, enclosureX )
-j Display with Primary / Secondary path information
-k Display the device status (Good/Failed)
-m List with Blockcount and Block Size ( max.LBA+1 )
-n Displays full WWNN and prevents display of column headings
-p Display with physical numbers/names (eg. pdiskX)
When an Adapter the physical name is the Hostname.
Hostnames that are greater than 15 characters will
be truncated, except when the -n is used.
When an Enclosure the physical name is the
four character Enclosure Name
-q Displays configuration validity information per adapter
and summaries for DualLoops and Network.
-s Display Drive Size/Capacity (eg. 36GB)
-t Display Drive Types (eg. DGHC)
-x Display Drive Serial Number (eg. 80004S60E7DD20D)
Read issmap output
26
Read issmap output (Continue)
27
In certern cases, you can see d instead of c, which means that port of DDM is disabled.
Changes to issmap output in DS8800
28
cyan tool
cyan shows the backend loop connection, which provide a fast way for remote support to understand the backend loop topology. It can be run both on pe package and live machine. But do NOT run live cyan in the machine when SSD drives are installed.
[iss002/iss015-1]-[RJ0BHDY/R1-S23/enclosure1 ]-[RJ0BGPJ/R1-S24/enclosure0 ]
[iss002/iss015-2]-[RJ0BJ2J/R1-S13/enclosure3 ]-[RJ0BHGC/R1-S14/enclosure2 ]
[iss022/iss035-1]-[RJ0BGMW/R1-S21/enclosure4 ]-[RJ0BJN0/R1-S22/enclosure6 ]
[iss022/iss035-2]-[RJ0BJNK/R1-S11/enclosure7 ]-[RJ0BHYH/R1-S12/enclosure5 ]
[iss062/iss075-1]-[RJ0BGJK/R2-S21/enclosure8 ]
[iss062/iss075-2]-[RJ0BGJM/R2-S11/enclosure9 ]
29
Information
Inforcenter
Squadrons-S PE Support:
DS8000 Problem Determination Education - Intermediate - FC-AL v 1.1.2
30
