Hitachi GAD(Global-Active Device)日常维护手册

Global-Active Device (GAD) 日常故障维护手册

Hitachi Data Systems
CS&S North China
修订 | 日期 | 修订人 | 描述 |
注意/说明
GAD 状态说明
对于一些故障无法预知在故障发生后哪个存储处于正常状态
例如仲裁盘(主/备)发生故障时,无法判断此时由哪台存储(主/备)接管业务IO,因此需要谨慎判断后才能选择正确的方法恢复GAD状态。
通常通过pair状态中的"L/M"mode来判断当前哪个卷正在接受IO
"L/M"mode表示GAD Pair状态时该卷可用(平时正常状态)
“L/L”mode表示发生故障时该卷可用
"B/B"mode表示发生故障时该卷不可用
下面的例子中因为P-VOL状态为”L/L”,所以此时P-VOL接收IO,S-VOL无法使用:
C:\Windows>pairdisplay -g SC-XLAB1E -fxce -IH48
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
SC-XLAB1E 007001-007001(L) (CL1-A-2, 1, 4)350216 7001.P-VOL PSUE NEVER , 100 7001 - - 0 2 - - - L/L
SC-XLAB1E 007001-007001(R) (CL3-A-2, 0, 1)350232 7001.S-VOL PSUE NEVER , 100 7001 - - 0 2 - - - B/B
SC-XLAB1E 007000-007000(L) (CL1-A-2, 1, 3)350216 7000.P-VOL PSUE NEVER , 9 7000 - - 0 2 - - - L/L
SC-XLAB1E 007000-007000(R) (CL3-A-2, 0, 0)350232 7000.S-VOL PSUE NEVER , 100 7000 - - 0 2 - - - B/B
下面的例子中因为S-VOL状态为”L/L”,所以此时S-VOL接收IO,P-VOL无法使用:
C:\Windows>pairdisplay -g SC-XLAB1E -fxce -IH48
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
SC-XLAB1E 007001-007001(L) (CL1-A-2, 1, 4)350216 7001.P-VOL PSUE NEVER , 100 7001 - - 0 2 - - - B/B
SC-XLAB1E 007001-007001(R) (CL3-A-2, 0, 1)350232 7001.S-VOL PSUE NEVER , 100 7001 - - 0 2 - - - L/L
SC-XLAB1E 007000-007000(L) (CL1-A-2, 1, 3)350216 7000.P-VOL PSUE NEVER , 100 7000 - - 0 2 - - - B/B
SC-XLAB1E 007000-007000(R) (CL3-A-2, 0, 0)350232 7000.S-VOL PSUE NEVER , 9 7000 - - 0 2 - - - L/L
当发生中裁盘(主/备)故障时,仲裁盘恢复为normal状态后仍然需要再等待5分钟,Pair同步的命令才能正常执行。
常见Pair状态说明
- “PAIR”,表示主/备存储复制卷正常同步状态
- “PSUS/SSUS”表示主/备存储的复制卷处于挂起状态(此种挂起属于人为干预的挂起而非故障)
- “PSUE”表示主/备存储复制卷处于挂起状态(此种挂起属于由故障导致的挂起,而非人为干预,属于异常状态)
- “COPY”,表示主/备存储卷正在执行数据同步(复制可能是创建Pair时初始拷贝,也可能是故障恢复后的增量拷贝)
- “SMPL”,表示主/备存储卷未建立Pair关系
CCI工具说明
CCI工具通过给存储发送指令来控制主/备存储间复制关系
CCI部署:目前站点 A (一中心)及站点 B (二中心)均各部署了1台CCI服务器,需注意CCI管理主/备存储间的复制方向。
CCI中的角色:
CCI中定义了主存储卷用”P-VOL”表示
CCI中定义了备存储卷用”S-VOL”表示
CCI实例说明:
CCI中需要为主/备存储卷各分配1个实例号使用
站点 A 到站点 B 的复制:
生产卷实例号:horcm100
备份卷实例号:horcm200
站点 B 到站点 A 的复制
生产卷实例号:horcm10
备份卷实例号:horcm20
站点 A CCI(IP: x.x.x.x,administrator/ <密码>):
管理站点 A VSPG1K(SN:xxxxx)到站点 B VSPG1K(SN:xxxxx)方向的复制。
此时两中心存储中卷的角色为:
站点 A VSPG1K(SN:xxxxx)卷为P-VOL(生产卷)
站点 B VSPG1K(SN:xxxxx)卷为S-VOL(备份卷)
站点 B CCI(IP: x.x.x.x,administrator/ <密码>):
管理站点 B VSPG1K(SN:xxxxx)到站点 A VSPG1K(SN:xxxxx)方向的复制。
此时两中心存储中卷的角色为:
站点 B VSPG1K(SN:xxxxx)卷为P-VOL(生产卷)
站点 A VSPG1K(SN:xxxxx)卷为S-VOL(备份卷)
GAD Pair状态检查方法
以检查站点 A VSPG1K(SN:xxxxx)到站点 B VSPG1K(SN:xxxxx)方向的复制为例:
登陆站点 A CCI服务器IP: x.x.x.x,administrator/ <密码>)
执行下述命令:
C:\Windows>pairdisplay -g TDB_A1 -fxce –CLI -IH100
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
TDB_A1 000200-001200(L) (CL1-A-2, 1, 4)350532 0200.P-VOL PAIR NEVER , 100 1200 - - 0 2 - - - L/M
TDB_A1 001200-000200(R) (CL3-A-2, 0, 1)350557 1200.S-VOL PAIR NEVER , 100 0200 - - 0 2 - - - L/M
以检查站点 B VSPG1K(SN:xxxxx)到站点 A VSPG1K(SN:xxxxx)方向的复制为例:
登陆站点 B CCI服务器IP: x.x.x.x,administrator/ <密码>)
执行下述命令:
C:\Windows>pairdisplay -g TKY_A1 -fxce –CLI -IH10
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
TKY_A1 000100-001100(L) (CL1-A-2, 1, 4)350557 0100.P-VOL PAIR NEVER , 100 1100 - - 0 2 - - - L/M
TKY_A1 001100-000100(R) (CL3-A-2, 0, 1)350532 1100.S-VOL PAIR NEVER , 100 0100 - - 0 2 - - - L/M
GAD Pair人为挂起方法
以检查站点 A VSPG1K(SN:xxxxx)到站点 B VSPG1K(SN:xxxxx)方向的复制为例:
登陆站点 A CCI服务器IP: x.x.x.x,administrator/ <密码>)
执行下述命令:
C:\Windows>pairsplit -g TDB_A1 -IH100
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
TDB_A1 000200-001200(L) (CL1-A-2, 1, 4)350532 0200.P-VOL PSUS NEVER , 98 1200 - - 0 2 - - - L/L
TDB_A1 001200-000200(R) (CL3-A-2, 0, 1)350557 1200.S-VOL SSUS NEVER , 100 0200 - - 0 2 - - - B/B
以检查站点 B VSPG1K(SN:xxxxx)到站点 A VSPG1K(SN:xxxxx)方向的复制为例:
登陆站点 B CCI服务器IP: x.x.x.x,administrator/ <密码>)
执行下述命令:
C:\Windows>pairsplit -g TKY_A1 -IH10
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
TKY_A1 000100-001100(L) (CL1-A-2, 1, 4)350557 0100.P-VOL PSUS NEVER , 98 1100 - - 0 2 - - - L/L
TKY_A1 001100-000100(R) (CL3-A-2, 0, 1)350532 1100.S-VOL SSUS NEVER , 100 0100 - - 0 2 - - - B/B
常见的几种Pair故障及恢复方法
故障1:当Pair挂起后(由故障引起,此时生产卷P-VOL状态为”L/L”)
生产存储卷(P-VOL)状态为”L/L”
备份存储卷(S-VOL)状态为”B/B”
以站点 B VSPG1K(SN:xxxxx)到站点 A VSPG1K(SN:xxxxx)方向的复制为例:
登陆站点 B CCI服务器IP: x.x.x.x,administrator/ <密码>)
执行下述命令:
C:\Windows>pairdisplay -g TKY_A1 -fxce –CLI -IH10
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
TKY_A1 000100-001100(L) (CL1-A-2, 1, 4)350557 0100.P-VOL PSUE NEVER , 100 1100 - - 0 2 - - - L/L
TKY_A1 001100-000100(R) (CL3-A-2, 0, 1)350532 1100.S-VOL PAIR NEVER , 100 0100 - - 0 2 - - - B/B
此时由于生产卷(P-VOL)状态为”L/L”,当前业务IO使用的是P-VOL,可采用正常pariresync同步方法恢复Pair状态:
恢复步骤:
- 确认主/备存储自身状态正常
- 确认主/备存储仲裁设备状态正常
- 确认主/备存储间远程链路状态正常
- 执行Pair同步命令pairresync恢复Pair同步
C:\Windows>pairresync -g TKY_A1 -fxce -IH10
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
TKY_A1 000100-001100(L) (CL1-A-2, 1, 4)350557 0100.P-VOL COPY NEVER , 92 1100 - - 0 2 - - - L/L
TKY_A1 001100-000100(R) (CL3-A-2, 0, 1)350532 1100.S-VOL COPY NEVER , 100 0100 - - 0 2 - - - B/B
- 等待同步完成达到”PAIR”状态后,确认P-VOL/S-VOL状态均变为”L/M”表示GAD Pair已恢复正常,显示如下
C:\Windows>pairdisplay -g TKY_A1 -fxce –CLI -IH10
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
TKY_A1 000100-001100(L) (CL1-A-2, 1, 4)350557 0100.P-VOL PAIR NEVER , 100 1100 - - 0 2 - - - L/M
TKY_A1 001100-000100(R) (CL3-A-2, 0, 1)350532 1100.S-VOL PAIR NEVER , 100 0100 - - 0 2 - - - L/M
故障2:当Pair挂起后(由故障引起,此时备份卷S-VOL状态为”L/L”)
生产存储卷(P-VOL)状态为”B/B”
备份存储卷(S-VOL)状态为”L/L”
以站点 B VSPG1K(SN:xxxxx)到站点 A VSPG1K(SN:xxxxx)方向的复制为例:
登陆站点 B CCI服务器IP: x.x.x.x,administrator/ <密码>)
执行下述命令:
C:\Windows>pairdisplay -g TKY_A1 -fxce –CLI -IH10
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
TKY_A1 000100-001100(L) (CL1-A-2, 1, 4)350557 0100.P-VOL PSUE NEVER , 100 1100 - - 0 2 - - - B/B
TKY_A1 001100-000100(R) (CL3-A-2, 0, 1)350532 1100.S-VOL PAIR NEVER , 100 0100 - - 0 2 - - - L/L
此时由于备份卷(S-VOL)状态为”L/L”,当前业务IO使用的是S-VOL,需采用pariresync --swaps同步方法恢复Pair状态:
恢复步骤:
- 确认主/备存储自身状态正常
- 确认主/备存储仲裁设备状态正常
- 确认主/备存储间远程链路状态正常
- 执行Pair同步命令pairresync -swaps恢复Pair同步
C:\Windows>pairresync -g TKY_A1 -fxce -swaps -IH20 (此命令需在备份存储CCI执行,此时需登录站点 A CCI执行命令)
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
TKY_A1 000100-001100(L) (CL1-A-2, 1, 4)350557 0100.S-VOL COPY NEVER , 92 1100 - - 0 2 - - - L/L
TKY_A1 001100-000100(R) (CL3-A-2, 0, 1)350532 1100.P-VOL COPY NEVER , 100 0100 - - 0 2 - - - B/B
- 等待同步完成达到”PAIR”状态后,确认P-VOL/S-VOL状态均变为”L/M”,显示如下
C:\Windows>pairdisplay -g TKY_A1 -fxce –CLI –IH20
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
TKY_A1 000100-001100(L) (CL1-A-2, 1, 4)350557 0100.S-VOL PAIR NEVER , 100 1100 - - 0 2 - - - L/M
TKY_A1 001100-000100(R) (CL3-A-2, 0, 1)350532 1100.P-VOL PAIR NEVER , 100 0100 - - 0 2 - - - L/M
- 再执行Pair同步命令pairresync -swaps恢复Pair同步
C:\Windows>pairresync -g TKY_A1 -fxce -swaps -IH10 (此命令需在主存储CCI执行,此时需登录站点 B CCI执行命令)
- 在站点 B CCI检查Pair状态及角色关系是否正确,如下所示:
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
TKY_A1 000100-001100(L) (CL1-A-2, 1, 4)350557 0100.P-VOL PAIR NEVER , 100 1100 - - 0 2 - - - L/M
TKY_A1 001100-000100(R) (CL3-A-2, 0, 1)350532 1100.S-VOL PAIR NEVER , 100 0100 - - 0 2 - - - L/M
故障3:当Pair挂起后(由故障引起,此时生产卷P-VOL/备份卷S-VOL状态为”B/B”)
生产存储卷(P-VOL)状态为”B/B”
备份存储卷(S-VOL)状态为”B/B”
以站点 B VSPG1K(SN:xxxxx)到站点 A VSPG1K(SN:xxxxx)方向的复制为例:
登陆站点 B CCI服务器IP: x.x.x.x,administrator/ <密码>)
执行下述命令:
C:\Windows>pairdisplay -g TKY_A1 -fxce –CLI -IH10
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
TKY_A1 000100-001100(L) (CL1-A-2, 1, 4)350557 0100.P-VOL PSUE NEVER , 100 1100 - - 0 2 - - - B/B
TKY_A1 001100-000100(R) (CL3-A-2, 0, 1)350532 1100.S-VOL SSUE NEVER , 100 0100 - - 0 2 - - - B/B
此时由于生产卷P-VOL及备份卷S-VOL状态均为”B/B”,GAD 完全失效,需要彻底解除GAD Pair关系,重建Pair才可恢复正常。
恢复步骤:
- 确认主/备存储自身状态正常
- 确认主/备存储仲裁设备状态正常
- 确认主/备存储间远程链路状态正常
- 彻底删除Pair关系,执行下述命令 :
C:\Windows>pairsplit –S -g TKY_A1 -fxce –CLI -IH10
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
TKY_A1 000100-001100(L) (CL1-A-2, 1, 4)350557 0100.P-VOL SMPL NEVER , - 1100 - - 0 2 - - - --
TKY_A1 001100-000100(R) (CL3-A-2, 0, 1)350532 1100.S-VOL SMPL NEVER , - 0100 - - 0 2 - - - ---
- 重建Pair关系,执行下述命令:
C:\Windows>paircreate –vl –f never –jq 1 -g TKY_A1 -IH10
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
TKY_A1 000100-001100(L) (CL1-A-2, 1, 4)350557 0100.P-VOL COPY NEVER , - 1100 - - 0 2 - - - --
TKY_A1 001100-000100(R) (CL3-A-2, 0, 1)350532 1100.S-VOL COPY NEVER , - 0100 - - 0 2 - - - ---
- 检查恢复Pair状态即可:
C:\Windows>pairdisplay -g TKY_A1 -fxce –CLI -IH10
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
TKY_A1 000100-001100(L) (CL1-A-2, 1, 4)350557 0100.P-VOL PAIR NEVER , 100 1100 - - 0 2 - - - L/M
TKY_A1 001100-000100(R) (CL3-A-2, 0, 1)350532 1100.S-VOL PAIR NEVER , 100 0100 - - 0 2 - - - L/M
几种GAD常见故障场景
备存储故障
- 故障状态(主卷P-VOL状态”L/L”,备卷S-VOL状态”B/B”)
C:\Windows>pairdisplay -g SC-XLAB1E -fxce -IH48
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
SC-XLAB1E 007001-007001(L) (CL1-A-2, 1, 4)350216 7001.P-VOL PSUE NEVER , 100 7001 - - 0 2 - - - L/L
SC-XLAB1E 007001-007001(R) (CL3-A-2, 0, 1)350232 7001.S-VOL PAIR NEVER , 100 7001 - - 0 2 - - - B/B
SC-XLAB1E 007000-007000(L) (CL1-A-2, 1, 3)350216 7000.P-VOL PSUE NEVER , 9 7000 - - 0 2 - - - L/L
SC-XLAB1E 007000-007000(R) (CL3-A-2, 0, 0)350232 7000.S-VOL PAIR NEVER , 100 7000 - - 0 2 - - - B/B
- 恢复步骤:
恢复中裁盘(备)状态为normal
恢复主/备存储之间的Remote Path状态为normal
使用pairresync命令恢复同步pair(须注意此时P-VOL状态为"L/L",正常执行同步命令即可)
C:\Windows>pairresync -g SC-XLAB1E -IH48
C:\Windows>pairdisplay -g SC-XLAB1E -fxce -IH48
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
SC-XLAB1E 007001-007001(L) (CL1-A-2, 1, 4)350216 7001.P-VOL PAIR NEVER , 100 7001 - - 0 2 - - - L/M
SC-XLAB1E 007001-007001(R) (CL3-A-2, 0, 1)350232 7001.S-VOL PAIR NEVER , 100 7001 - - 0 2 - - - L/M
SC-XLAB1E 007000-007000(L) (CL1-A-2, 1, 3)350216 7000.P-VOL COPY NEVER , 9 7000 - - 0 2 - - - L/M
SC-XLAB1E 007000-007000(R) (CL3-A-2, 0, 0)350232 7000.S-VOL COPY NEVER , 100 7000 - - 0 2 - - - B/B
C:\Windows>pairdisplay -g SC-XLAB1E -fxce -IH48
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
SC-XLAB1E 007001-007001(L) (CL1-A-2, 1, 4)350216 7001.P-VOL PAIR NEVER , 100 7001 - - 0 2 - - - L/M
SC-XLAB1E 007001-007001(R) (CL3-A-2, 0, 1)350232 7001.S-VOL PAIR NEVER , 100 7001 - - 0 2 - - - L/M
SC-XLAB1E 007000-007000(L) (CL1-A-2, 1, 3)350216 7000.P-VOL PAIR NEVER , 100 7000 - - 0 2 - - - L/M
SC-XLAB1E 007000-007000(R) (CL3-A-2, 0, 0)350232 7000.S-VOL PAIR NEVER , 100 7000 - - 0 2 - - - L/M
主存储故障
- 故障状态
此时备卷S-VOL状态为”L/L”,此时S-VOL接收IO:
C:\Windows>pairdisplay -g SC-XLAB1E -fxce –l -IH49
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
SC-XLAB1E 007001-007001(L) (CL3-A-2, 0, 1)350232 7001.S-VOL SSWS NEVER , 100 7001 - - 0 2 - - - L/L
SC-XLAB1E 007000-007000(L) (CL3-A-2, 0, 0)350232 7000.S-VOL SSWS NEVER , 97 7000 - - 0 2 - - - L/L
- 恢复步骤
- 恢复中裁盘(备)状态为normal
- 恢复主/备存储之间的Remote Path状态为normal
- 在主存储的CCI上执行Pairdisplay查看pair状态显示如下:
C:\Windows>pairdisplay -g SC-XLAB1E -fxce -IH48
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
SC-XLAB1E 007001-007001(L) (CL1-A-2, 1, 4)350216 7001.P-VOL PSUE NEVER , 100 7001 - - 0 2 - - - B/B
SC-XLAB1E 007001-007001(R) (CL3-A-2, 0, 1)350232 7001.S-VOL SSWS NEVER , 100 7001 - - 0 2 - - - L/L
SC-XLAB1E 007000-007000(L) (CL1-A-2, 1, 3)350216 7000.P-VOL PSUE NEVER , 100 7000 - - 0 2 - - - B/B
SC-XLAB1E 007000-007000(R) (CL3-A-2, 0, 0)350232 7000.S-VOL SSWS NEVER , 9 7000 - - 0 2 - - - L/L
使用pairresync命令恢复同步pair,须注意此时S-VOL状态为"L/L",需要执行swaps同步命令即可
C:\Windows>pairresync -g SC-XLAB1E -swaps -IH49 ------ (此命令需要在备存储CCI执行)
C:\Windows>pairdisplay -g SC-XLAB1E -fxce -IH49
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
SC-XLAB1E 007001-007001(L) (CL1-A-2, 1, 4)350216 7001.S-VOL PAIR NEVER , 100 7001 - - 0 2 - - - L/M
SC-XLAB1E 007001-007001(R) (CL3-A-2, 0, 1)350232 7001.P-VOL PAIR NEVER , 100 7001 - - 0 2 - - - L/M
SC-XLAB1E 007000-007000(L) (CL1-A-2, 1, 3)350216 7000.S-VOL PAIR NEVER , 100 7000 - - 0 2 - - - L/M
SC-XLAB1E 007000-007000(R) (CL3-A-2, 0, 0)350232 7000.P-VOL PAIR NEVER , 100 7000 - - 0 2 - - - L/M
C:\Windows>pairresync -g SC-XLAB1E -swaps -IH48 ------ (此命令需要在主存储CCI执行)
C:\Windows>pairdisplay -g SC-XLAB1E -fxce -IH48
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
SC-XLAB1E 007001-007001(L) (CL1-A-2, 1, 4)350216 7001.P-VOL PAIR NEVER , 100 7001 - - 0 2 - - - L/M
SC-XLAB1E 007001-007001(R) (CL3-A-2, 0, 1)350232 7001.S-VOL PAIR NEVER , 100 7001 - - 0 2 - - - L/M
SC-XLAB1E 007000-007000(L) (CL1-A-2, 1, 3)350216 7000.P-VOL PAIR NEVER , 100 7000 - - 0 2 - - - L/M
SC-XLAB1E 007000-007000(R) (CL3-A-2, 0, 0)350232 7000.S-VOL PAIR NEVER , 100 7000 - - 0 2 - - - L/M
主存储的仲裁盘故障
注意:
对于一些故障无法预知在故障发生后哪个存储处于正常状态
例如仲裁盘(主/备)发生故障时,无法判断此时由哪台存储接管业务IO,因此需要谨慎判断后才能选择正确的发生恢复GAD状态。
通常通过pair状态中的"L/M"mode来判断当前哪个卷正在接受IO
"L/M"mode表示该卷正在接收IO
"B/B"mode表示该卷不接收IO
- 故障状态:
- 此时P-VOL状态为”L/L”,S-VOL状态为”B/B”
C:\Windows>pairdisplay -g SC-XLAB1E -fxce -IH48
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
SC-XLAB1E 007001-007001(L) (CL1-A-2, 1, 4)350216 7001.P-VOL PSUE NEVER , 100 7001 - - 0 2 - - - L/L
SC-XLAB1E 007001-007001(R) (CL3-A-2, 0, 1)350232 7001.S-VOL PSUE NEVER , 100 7001 - - 0 2 - - - B/B
SC-XLAB1E 007000-007000(L) (CL1-A-2, 1, 3)350216 7000.P-VOL PSUE NEVER , 9 7000 - - 0 2 - - - L/L
SC-XLAB1E 007000-007000(R) (CL3-A-2, 0, 0)350232 7000.S-VOL PSUE NEVER , 100 7000 - - 0 2 - - - B/B
- 恢复步骤:
- 恢复主存储到仲裁存储的链路
- 仲裁盘如果状态为blocked 需要先在仲裁存储内将仲裁盘格式化,还需在主存储中将其格式化.
- 确认主存储的仲裁盘状态为normal
- Pairdisplay output检查pair状态,P-VOL为”L/L”
C:\Windows>pairdisplay -g SC-XLAB1E -fxce -IH48
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
SC-XLAB1E 007001-007001(L) (CL1-A-2, 1, 4)350216 7001.P-VOL PSUE NEVER , 100 7001 - - 0 2 - - - L/L
SC-XLAB1E 007001-007001(R) (CL3-A-2, 0, 1)350232 7001.S-VOL PSUE NEVER , 100 7001 - - 0 2 - - - B/B
SC-XLAB1E 007000-007000(L) (CL1-A-2, 1, 3)350216 7000.P-VOL PSUE NEVER , 9 7000 - - 0 2 - - - L/L
SC-XLAB1E 007000-007000(R) (CL3-A-2, 0, 0)350232 7000.S-VOL PSUE NEVER , 100 7000 - - 0 2 - - - B/B
- 在主存储的CCI中执行正常的同步命令pairresync即可:
C:\Windows>pairresync -g SC-XLAB1E -IH48
C:\Windows>pairdisplay -g SC-XLAB1E -fxce -IH48
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
SC-XLAB1E 007001-007001(L) (CL1-A-2, 1, 4)350216 7001.P-VOL PAIR NEVER , 100 7001 - - 0 2 - - - L/M
SC-XLAB1E 007001-007001(R) (CL3-A-2, 0, 1)350232 7001.S-VOL PAIR NEVER , 100 7001 - - 0 2 - - - L/M
SC-XLAB1E 007000-007000(L) (CL1-A-2, 1, 3)350216 7000.P-VOL COPY NEVER , 9 7000 - - 0 2 - - - L/M
SC-XLAB1E 007000-007000(R) (CL3-A-2, 0, 0)350232 7000.S-VOL COPY NEVER , 100 7000 - - 0 2 - - - B/B
C:\Windows>pairdisplay -g SC-XLAB1E -fxce -IH48
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
SC-XLAB1E 007001-007001(L) (CL1-A-2, 1, 4)350216 7001.P-VOL PAIR NEVER , 100 7001 - - 0 2 - - - L/M
SC-XLAB1E 007001-007001(R) (CL3-A-2, 0, 1)350232 7001.S-VOL PAIR NEVER , 100 7001 - - 0 2 - - - L/M
SC-XLAB1E 007000-007000(L) (CL1-A-2, 1, 3)350216 7000.P-VOL PAIR NEVER , 100 7000 - - 0 2 - - - L/M
SC-XLAB1E 007000-007000(R) (CL3-A-2, 0, 0)350232 7000.S-VOL PAIR NEVER , 100 7000 - - 0 2 - - - L/M
备存储仲裁盘故障
注意:
对于一些故障无法预知在故障发生后哪个存储处于正常状态
例如仲裁盘(主/备)发生故障时,无法判断此时由哪台存储接管业务IO,因此需要谨慎判断后才能选择正确的发生恢复GAD状态。
通常通过pair状态中的"L/M"mode来判断当前哪个卷正在接受IO
"L/M"mode表示该卷正在接收IO
"B/B"mode表示该卷不接收IO
- 故障状态:
- 此时P-VOL状态为”L/L”,S-VOL状态为”B/B”
C:\Windows>pairdisplay -g SC-XLAB1E -fxce -IH48
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
SC-XLAB1E 007001-007001(L) (CL1-A-2, 1, 4)350216 7001.P-VOL PSUE NEVER , 100 7001 - - 0 2 - - - L/L
SC-XLAB1E 007001-007001(R) (CL3-A-2, 0, 1)350232 7001.S-VOL PSUE NEVER , 100 7001 - - 0 2 - - - B/B
SC-XLAB1E 007000-007000(L) (CL1-A-2, 1, 3)350216 7000.P-VOL PSUE NEVER , 9 7000 - - 0 2 - - - L/L
SC-XLAB1E 007000-007000(R) (CL3-A-2, 0, 0)350232 7000.S-VOL PSUE NEVER , 100 7000 - - 0 2 - - - B/B
- 恢复步骤:
- 恢复主存储到仲裁存储的链路
- 仲裁盘如果状态为blocked 需要先在仲裁存储内将仲裁盘格式化,还需在主存储中将其格式化.
- 确认主存储的仲裁盘状态为normal
- Pairdisplay output检查pair状态,P-VOL为”L/L”
C:\Windows>pairdisplay -g SC-XLAB1E -fxce -IH48
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
SC-XLAB1E 007001-007001(L) (CL1-A-2, 1, 4)350216 7001.P-VOL PSUE NEVER , 100 7001 - - 0 2 - - - L/L
SC-XLAB1E 007001-007001(R) (CL3-A-2, 0, 1)350232 7001.S-VOL PSUE NEVER , 100 7001 - - 0 2 - - - B/B
SC-XLAB1E 007000-007000(L) (CL1-A-2, 1, 3)350216 7000.P-VOL PSUE NEVER , 9 7000 - - 0 2 - - - L/L
SC-XLAB1E 007000-007000(R) (CL3-A-2, 0, 0)350232 7000.S-VOL PSUE NEVER , 100 7000 - - 0 2 - - - B/B
- 在主存储的CCI中执行正常的同步命令pairresync即可:
C:\Windows>pairresync -g SC-XLAB1E -IH48
C:\Windows>pairdisplay -g SC-XLAB1E -fxce -IH48
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
SC-XLAB1E 007001-007001(L) (CL1-A-2, 1, 4)350216 7001.P-VOL PAIR NEVER , 100 7001 - - 0 2 - - - L/M
SC-XLAB1E 007001-007001(R) (CL3-A-2, 0, 1)350232 7001.S-VOL PAIR NEVER , 100 7001 - - 0 2 - - - L/M
SC-XLAB1E 007000-007000(L) (CL1-A-2, 1, 3)350216 7000.P-VOL COPY NEVER , 9 7000 - - 0 2 - - - L/M
SC-XLAB1E 007000-007000(R) (CL3-A-2, 0, 0)350232 7000.S-VOL COPY NEVER , 100 7000 - - 0 2 - - - B/B
C:\Windows>pairdisplay -g SC-XLAB1E -fxce -IH48
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
SC-XLAB1E 007001-007001(L) (CL1-A-2, 1, 4)350216 7001.P-VOL PAIR NEVER , 100 7001 - - 0 2 - - - L/M
SC-XLAB1E 007001-007001(R) (CL3-A-2, 0, 1)350232 7001.S-VOL PAIR NEVER , 100 7001 - - 0 2 - - - L/M
SC-XLAB1E 007000-007000(L) (CL1-A-2, 1, 3)350216 7000.P-VOL PAIR NEVER , 100 7000 - - 0 2 - - - L/M
SC-XLAB1E 007000-007000(R) (CL3-A-2, 0, 0)350232 7000.S-VOL PAIR NEVER , 100 7000 - - 0 2 - - - L/M
主/备所有仲裁盘
注意:
对于一些故障无法预知在故障发生后哪个存储处于正常状态
例如仲裁盘(主/备)发生故障时,无法判断此时由哪台存储接管业务IO,因此需要谨慎判断后才能选择正确的发生恢复GAD状态。
通常通过pair状态中的"L/M"mode来判断当前哪个卷正在接受IO
"L/M"mode表示该卷正在接收IO
"B/B"mode表示该卷不接收IO
- 故障状态:
- 此时P-VOL状态为”L/L”,S-VOL状态为”B/B”
C:\Windows>pairdisplay -g SC-XLAB1E -fxce -IH48
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
SC-XLAB1E 007001-007001(L) (CL1-A-2, 1, 4)350216 7001.P-VOL PSUE NEVER , 100 7001 - - 0 2 - - - L/L
SC-XLAB1E 007001-007001(R) (CL3-A-2, 0, 1)350232 7001.S-VOL PSUE NEVER , 100 7001 - - 0 2 - - - B/B
SC-XLAB1E 007000-007000(L) (CL1-A-2, 1, 3)350216 7000.P-VOL PSUE NEVER , 9 7000 - - 0 2 - - - L/L
SC-XLAB1E 007000-007000(R) (CL3-A-2, 0, 0)350232 7000.S-VOL PSUE NEVER , 100 7000 - - 0 2 - - - B/B
- 恢复步骤:
- 恢复主存储到仲裁存储的链路
- 仲裁盘如果状态为blocked 需要先在仲裁存储内将仲裁盘格式化,还需在主存储中将其格式化.
- 确认主存储的仲裁盘状态为normal
- Pairdisplay output检查pair状态,P-VOL为”L/L”
C:\Windows>pairdisplay -g SC-XLAB1E -fxce -IH48
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
SC-XLAB1E 007001-007001(L) (CL1-A-2, 1, 4)350216 7001.P-VOL PSUE NEVER , 100 7001 - - 0 2 - - - L/L
SC-XLAB1E 007001-007001(R) (CL3-A-2, 0, 1)350232 7001.S-VOL PSUE NEVER , 100 7001 - - 0 2 - - - B/B
SC-XLAB1E 007000-007000(L) (CL1-A-2, 1, 3)350216 7000.P-VOL PSUE NEVER , 9 7000 - - 0 2 - - - L/L
SC-XLAB1E 007000-007000(R) (CL3-A-2, 0, 0)350232 7000.S-VOL PSUE NEVER , 100 7000 - - 0 2 - - - B/B
- 在主存储的CCI中执行正常的同步命令pairresync即可:
C:\Windows>pairresync -g SC-XLAB1E -IH48
C:\Windows>pairdisplay -g SC-XLAB1E -fxce -IH48
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
SC-XLAB1E 007001-007001(L) (CL1-A-2, 1, 4)350216 7001.P-VOL PAIR NEVER , 100 7001 - - 0 2 - - - L/M
SC-XLAB1E 007001-007001(R) (CL3-A-2, 0, 1)350232 7001.S-VOL PAIR NEVER , 100 7001 - - 0 2 - - - L/M
SC-XLAB1E 007000-007000(L) (CL1-A-2, 1, 3)350216 7000.P-VOL COPY NEVER , 9 7000 - - 0 2 - - - L/M
SC-XLAB1E 007000-007000(R) (CL3-A-2, 0, 0)350232 7000.S-VOL COPY NEVER , 100 7000 - - 0 2 - - - B/B
C:\Windows>pairdisplay -g SC-XLAB1E -fxce -IH48
Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence, %,P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W
SC-XLAB1E 007001-007001(L) (CL1-A-2, 1, 4)350216 7001.P-VOL PAIR NEVER , 100 7001 - - 0 2 - - - L/M
SC-XLAB1E 007001-007001(R) (CL3-A-2, 0, 1)350232 7001.S-VOL PAIR NEVER , 100 7001 - - 0 2 - - - L/M
SC-XLAB1E 007000-007000(L) (CL1-A-2, 1, 3)350216 7000.P-VOL PAIR NEVER , 100 7000 - - 0 2 - - - L/M
SC-XLAB1E 007000-007000(R) (CL3-A-2, 0, 0)350232 7000.S-VOL PAIR NEVER , 100 7000 - - 0 2 - - - L/M
主存储和主存储仲裁盘同时故障
- 如果P-VOL与S-VOL同时为"B/B"状态,唯一解决方法是删除GAD pair,重新创建GAD pair
- Delete GAD pairs
- This must be done in Storage Navigator (only way to use the force option)
- Storage Navigator can be accessed using the emergency link or by right clicking on the array in HCS and selecting Remote Connections
- Select the GAD pairs that are reporting P-Vols and S-Vols blocked
- Select More Options and then Delete Pairs
- Note: by default the Delete Mode is set to Force. In the confirmation screen to delete pairs nothing should need to be changed.
- Click Finish