正文文本
联想专业服务
GSS 存储硬盘更换
1. 硬盘更换步骤概述
GSS 存储硬盘更换步骤如下：
1、 硬件状态查看
2、 文件系统及recovery group 状态查看
3、 确认需要更换的磁盘
4、 释放磁盘
5、 物理更换磁盘
6、 在系统中更换磁盘
7、 查看状态，确认更换完成
2. 硬盘更换详细说明
2.1. 硬件状态查看
确认控制器（x3650 服务器）状态正常，无硬件告警，记录有黄灯告警的磁盘柜和磁盘位置，
例如e2d3s05。
GSS22、GSS24、GSS26 使用4U60 盘位的磁盘柜，与原IBM 型号为DCS3700 扩展柜相同，没
有控制器，只有ESM。
磁盘柜称为enclosure，通常由下至上从1 开始编号，最下方为e1。每个磁盘柜有五个磁盘
抽屉，称为drawer，由下至上从1 开始编号，最下方为d1。每个抽屉含有12 块硬盘，编号
方式如下图。
1
联想专业服务
除enclosure 1 的drawer1 和5 分别安装1 块200GB SSD 外，其余enclosure 相应位置空缺，
每个enclosure 含有58 块6TB NL SAS 7200rpm 硬盘（11+12+12+12+11）。见下图
2.2. 文件系统和recovery group 状态查看
确认文件系统正常挂载；
#mmlsmount all
确认recovery group 状态正常，命令如下
[root@mgt03 ~]# mmlsrecoverygroup
declustered
arrays with
recovery group vdisks vdisks servers
------------------ ----------- ------ -------
gss01_opa1 3 5 gss01_opa1,gss02_opa1
gss02_opa1 3 5 gss02_opa1,gss01_opa1
gss03_opa1 3 5 gss03_opa1,gss04_opa1
gss04_opa1 3 5 gss04_opa1,gss03_opa1
高亮部分为recovery group 的名称，对于每个recovery group, 可以查看详细信息，如下。注
意查看磁盘状态
[root@mgt03 ~]# mmlsrecoverygroup gss01_opa1 -L --pdisk
declustered current allowable
recovery group arrays vdisks pdisks format version format version
----------------- ----------- ------ ------ -------------- --------------
gss01_opa1 3 5 119 4.2.0.1 4.2.0.1
declustered needs replace scrub background
2
联想专业服务
activity
array service vdisks pdisks spares threshold free space duration task progress priority
----------- ------- ------ ------ ------ --------- ---------- -------- -------------------------
SSD no 1 1 0,0 1 186 GiB 14 days scrub 8%
low
NVR no 1 2 0,0 1 3632 MiB 14 days scrub
16% low
DA1 no 3 116 4,60 1 9256 GiB 14 days scrub 76%
low
declustered checksum
vdisk RAID code array vdisk size block size granularity state
remarks
------------------ ------------------ ----------- ---------- ---------- ----------- ----- -------
gss01_opa1_logtip 2WayReplication NVR 48 MiB 2 MiB 512 ok
logTip
gss01_opa1_logTipBackup Unreplicated SSD 48 MiB 2 MiB 512
ok logTipBackup
gss01_opa1_loghome 4WayReplication DA1 20 GiB 2 MiB 512 ok
log
gss01_opa1_Data_16M_2p_1 8+2p DA1 479 TiB 16 MiB 32 KiB
ok
gss01_opa1_MetaData_16M_2p_1 3WayReplication DA1 103 GiB 1 MiB 32
KiB ok
config data declustered array VCD spares actual rebuild spare space remarks
------------------ ------------------ ------------- --------------------------------- ----------------
rebuild space DA1 60 64 pdisk
config data max disk group fault tolerance actual disk group fault tolerance remarks
------------------ --------------------------------- --------------------------------- ----------------
rg descriptor 1 enclosure + 1 drawer 1 enclosure + 1 drawer limiting fault
tolerance
system index 2 enclosure 1 enclosure + 1 drawer limited
by rg descriptor
vdisk max disk group fault tolerance actual disk group fault tolerance remarks
------------------ --------------------------------- --------------------------------- ----------------
gss01_opa1_logtip 1 pdisk 1 pdisk
gss01_opa1_logTipBackup 0 pdisk 0 pdisk
gss01_opa1_loghome 3 enclosure 1 enclosure + 1 drawer limited
by rg descriptor
gss01_opa1_Data_16M_2p_1 2 drawer 2 drawer
gss01_opa1_MetaData_16M_2p_1 2 enclosure 1 enclosure + 1 drawer
3
联想专业服务
limited by rg descriptor
active recovery group server servers
----------------------------------------------- -------
gss01_opa1 gss01_opa1,gss02_opa1
2.3. 确认需要更换的磁盘
使用下面的命令确认需要更换的磁盘，注意核对磁盘的位置。
[root@gss01_opa1 bin]# mmlspdisk all --not-ok
pdisk:
replacementPriority = 4.66
name = "e3d3s06"
device = ""
recoveryGroup = "gss01_opa1"
declusteredArray = "DA1"
state = "dead/noPath/systemDrain/noRGD/noVCD/noData/replace"
capacity = 6000069312512
freeSpace = 5995774345216
fru = "00NA572"
location = "SV64235560-3-6"
WWN = "naa.5000C50086952C87"
server = "gss01_opa1"
reads = 1
writes = 886
bytesReadInGiB = 0.002
bytesWrittenInGiB = 0.237
IOErrors = 1
IOTimeouts = 0
mediaErrors = 0
checksumErrors = 0
pathErrors = 0
relativePerformance = 0.996
dataBadness = 0.000
rgIndex = 77
userLocation = "Enclosure SV64235560 Drawer 3 Slot 6"
userCondition = "replaceable"
hardware = " "
hardwareType = Rotating 7200
nPaths = 0 active 0 total
nsdFormatVersion = Unknown
paxosAreaOffset = Unknown
paxosAreaSize = Unknown
4
联想专业服务
logicalBlockSize = 512
2.4. 释放需要更换的磁盘
运行如下命令释放需要更换的磁盘，注意，recovery group 和磁盘的名字需要替换成用户处
相应的名字。
[root@gss01_opa1 bin]# mmchcarrier gss01_opa1 --pdisk e3d3s06 --release
Suspending pdisk e3d3s06 of RG gss01_opa1 in location SV64235560-3-6.
Location SV64235560-3-6 is Enclosure SV64235560 Drawer 3 Slot 6.
Carrier released.
- Remove carrier.
- Replace disk in location SV64235560-3-6 with FRU 00NA572.
- Reinsert carrier.
- Issue the following command:
mmchcarrier gss01_opa1 --replace --pdisk 'e3d3s06'
命令完成后相应磁盘会亮起蓝色指示灯
2.5. 物理更换磁盘
取下磁盘柜面网，按照蓝色指示灯抽出磁盘抽屉，找到相应磁盘，抬起手柄，取出硬盘。
如果备件没有手柄，请将原有手柄两侧向外张开，取下手柄，安装在新硬盘上。
将新硬盘推入原位置，压下手柄，卡住硬盘。推回磁盘抽屉，确认两侧卡住。
观察状态指示灯，此时应该闪烁，正在识别硬盘。
2.6. 在系统中更换磁盘
使用上面指示的命令在系统中更换磁盘，注意实际的recovery group 和磁盘名称，例如
mmchcarrier gss01_opa1 --replace --pdisk 'e3d3s06'
如果磁盘损坏到硬盘更换期间GSS 系统曾经关机，特别是意外关机，上述命令有可能失败。
出现这种问题需要用其他命令加入磁盘，具体方法如下：
查看pdisk 信息，注意替换实际的recovery group 和pdisk 名称
5
联想专业服务
[root@gss01_opa1 by-path]# mmlspdisk gss01_opa1 --pdisk e4d3s05
pdisk:
replacementPriority = 1000
name = "e4d3s05"
device = "/dev/sdai,/dev/sdlz"
recoveryGroup = "gss01_opa1"
declusteredArray = "DA1"
state = "dead/noPath/systemDrain/noRGD/noVCD/noData/replace"
capacity = 6000069312512
freeSpace = 292057776128
fru = "00NA572"
location = "SV64039345-3-5"
WWN = "naa.5000C5008694BB9B"
server = "gss01_opa1"
reads = 1021972
writes = 1299377
bytesReadInGiB = 1991.110
bytesWrittenInGiB = 2484.095
IOErrors = 0
IOTimeouts = 0
mediaErrors = 0
checksumErrors = 0
pathErrors = 0
relativePerformance = 1.002
dataBadness = 0.000
rgIndex = 105
userLocation = "Enclosure SV64039345 Drawer 3 Slot 5"
userCondition = "normal"
hardware = "IBM-ESXS ST6000NM0034 X BC6A Z4D4ZJFB0000R727HSTK"
hardwareType = Rotating 7200
nPaths = 2 active 4 total
nsdFormatVersion = NSD version 2
paxosAreaOffset = 6001170800640
paxosAreaSize = 4194304
logicalBlockSize = 512
进入/dev/disk/by-path 路径，按照上述WWN 查找对应的设备文件
[root@gss01_opa1 by-path]# ls -l|grep -i 5000C5008694BB9
lrwxrwxrwx 1 root root 10 Sep 22 18:27 pci-0000:83:00.0-sas-0x5000c5008694bb99-lun-0
-> ../../sdai
lrwxrwxrwx 1 root root 11 Sep 22 18:27 pci-0000:83:00.0-sas-0x5000c5008694bb99-lun-0-part1
-> ../../sdai1
lrwxrwxrwx 1 root root 10 Sep 22 18:27 pci-0000:88:00.0-sas-0x5000c5008694bb9a-lun-0
6
联想专业服务
-> ../../sdlz
lrwxrwxrwx 1 root root 11 Sep 22 18:27 pci-0000:88:00.0-sas-0x5000c5008694bb9a-lun-0-part1
-> ../../sdlz1
[root@gss01_opa1 by-path]#
编辑含有如下内容的文件，注意文件中含有pdisk 名，在哪个节点上看到的设备文件名，多
路径取其中一个即可，不要写两行。
[root@gss03_opa1 setupTools]# cat stanza_e4d3s05
%pdisk: pdiskName=e4d3s05 device=//gss01_opa1/dev/sdai da=DA1
[root@gss03_opa1 setupTools]#
然后运行如下命令加入并替换磁盘，注意recovery group 名，pdisk 定义文件名，以及’replace’
选项。
mmaddpdisk gss01_opa1 -F Stanza_e4d3s05 --replace
2.7. 查看状态，确认更换完成
上述命令成功后，请检查状态，注意替换recovery group 和磁盘名称
[root@mgt03 ~]# mmlspdisk gss01_opa1 --pdisk e4d5s06
pdisk:
replacementPriority = 1000
name = "e4d5s06"
device = "/dev/sdbh,/dev/sdnu"
recoveryGroup = "gss01_opa1"
declusteredArray = "DA1"
state = "ok"
capacity = 6000069312512
freeSpace = 292057776128
fru = "00NA572"
location = "SV64039345-5-6"
WWN = "naa.5000C500869411CF"
server = "gss01_opa1"
reads = 861000
writes = 1280221
bytesReadInGiB = 1679.520
bytesWrittenInGiB = 2464.192
IOErrors = 0
IOTimeouts = 0
mediaErrors = 0
checksumErrors = 0
7
联想专业服务
pathErrors = 0
relativePerformance = 1.027
dataBadness = 0.000
rgIndex = 118
userLocation = "Enclosure SV64039345 Drawer 5 Slot 6"
userCondition = "normal"
hardware = "IBM-ESXS ST6000NM0034 X BC6A Z4D4V10F0000R724MFYD"
hardwareType = Rotating 7200
nPaths = 2 active 4 total
nsdFormatVersion = NSD version 2
paxosAreaOffset = 6001170800640
paxosAreaSize = 4194304
logicalBlockSize = 512
确认整个recovery group 的状态，注意替换recovery group 名字，注意观察磁盘状态是’ok’
[root@mgt03 ~]# mmlsrecoverygroup gss01_opa1 -L --pdisk
declustered current allowable
recovery group arrays vdisks pdisks format version format version
----------------- ----------- ------ ------ -------------- --------------
gss01_opa1 3 5 119 4.2.0.1 4.2.0.1
declustered needs replace scrub
background activity
array service vdisks pdisks spares threshold free space duration task
progress priority
----------- ------- ------ ------ ------ --------- ---------- -------- -------------------------
SSD no 1 1 0,0 1 186 GiB 14 days
scrub 8% low
NVR no 1 2 0,0 1 3632 MiB 14 days
scrub 16% low
DA1 no 3 116 4,60 1 9256 GiB 14 days
scrub 76% low
n. active, declustered user state,
pdisk total paths array free space condition remarks
----------------- ----------- ----------- ---------- ----------- -------
e1d1s01 2, 4 DA1 276 GiB normal ok
e1d1s02 2, 4 DA1 272 GiB normal ok
e1d1s03ssd 2, 4 SSD 186 GiB normal ok
e1d1s04 2, 4 DA1 272 GiB normal ok
e1d1s05 2, 4 DA1 276 GiB normal ok
e1d1s06 2, 4 DA1 276 GiB normal ok
e1d2s01 2, 4 DA1 272 GiB normal ok
8
联想专业服务
e1d2s02 2, 4 DA1 272 GiB normal ok
e1d2s03 2, 4 DA1 272 GiB normal ok
e1d2s04 2, 4 DA1 272 GiB normal ok
e1d2s05 2, 4 DA1 272 GiB normal ok
e1d2s06 2, 4 DA1 272 GiB normal ok
e1d3s01 2, 4 DA1 272 GiB normal ok
e1d3s02 2, 4 DA1 272 GiB normal ok
e1d3s03 2, 4 DA1 272 GiB normal ok
e1d3s04 2, 4 DA1 272 GiB normal ok
e1d3s05 2, 4 DA1 272 GiB normal ok
e1d3s06 2, 4 DA1 272 GiB normal ok
e1d4s01 2, 4 DA1 272 GiB normal ok
e1d4s02 2, 4 DA1 272 GiB normal ok
e1d4s03 2, 4 DA1 272 GiB normal ok
e1d4s04 2, 4 DA1 272 GiB normal ok
e1d4s05 2, 4 DA1 272 GiB normal ok
e1d4s06 2, 4 DA1 272 GiB normal ok
e1d5s01 2, 4 DA1 272 GiB normal ok
e1d5s02 2, 4 DA1 272 GiB normal ok
e1d5s03 2, 4 DA1 272 GiB normal ok
e1d5s04 2, 4 DA1 272 GiB normal ok
e1d5s05 2, 4 DA1 272 GiB normal ok
e1d5s06 2, 4 DA1 272 GiB normal ok
e2d1s01 2, 4 DA1 272 GiB normal ok
e2d1s02 2, 4 DA1 276 GiB normal ok
e2d1s04 2, 4 DA1 272 GiB normal ok
e2d1s05 2, 4 DA1 276 GiB normal ok
e2d1s06 2, 4 DA1 276 GiB normal ok
e2d2s01 2, 4 DA1 272 GiB normal ok
e2d2s02 2, 4 DA1 272 GiB normal ok
e2d2s03 2, 4 DA1 272 GiB normal ok
e2d2s04 2, 4 DA1 272 GiB normal ok
e2d2s05 2, 4 DA1 272 GiB normal ok
e2d2s06 2, 4 DA1 272 GiB normal ok
e2d3s01 2, 4 DA1 272 GiB normal ok
e2d3s02 2, 4 DA1 272 GiB normal ok
e2d3s03 2, 4 DA1 272 GiB normal ok
e2d3s04 2, 4 DA1 272 GiB normal ok
e2d3s05 2, 4 DA1 272 GiB normal ok
e2d3s06 2, 4 DA1 272 GiB normal ok
e2d4s01 2, 4 DA1 272 GiB normal ok
e2d4s02 2, 4 DA1 272 GiB normal ok
e2d4s03 2, 4 DA1 272 GiB normal ok
e2d4s04 2, 4 DA1 272 GiB normal ok
9
联想专业服务
e2d4s05 2, 4 DA1 272 GiB normal ok
e2d4s06 2, 4 DA1 272 GiB normal ok
e2d5s01 2, 4 DA1 272 GiB normal ok
e2d5s02 2, 4 DA1 272 GiB normal ok
e2d5s03 2, 4 DA1 272 GiB normal ok
e2d5s04 2, 4 DA1 272 GiB normal ok
e2d5s05 2, 4 DA1 272 GiB normal ok
e2d5s06 2, 4 DA1 272 GiB normal ok
e3d1s01 2, 4 DA1 276 GiB normal ok
e3d1s02 2, 4 DA1 272 GiB normal ok
e3d1s04 2, 4 DA1 272 GiB normal ok
e3d1s05 2, 4 DA1 276 GiB normal ok
e3d1s06 2, 4 DA1 276 GiB normal ok
e3d2s01 2, 4 DA1 272 GiB normal ok
e3d2s02 2, 4 DA1 272 GiB normal ok
e3d2s03 2, 4 DA1 272 GiB normal ok
e3d2s04 2, 4 DA1 272 GiB normal ok
e3d2s05 2, 4 DA1 272 GiB normal ok
e3d2s06 2, 4 DA1 272 GiB normal ok
e3d3s01 2, 4 DA1 272 GiB normal ok
e3d3s02 2, 4 DA1 272 GiB normal ok
e3d3s03 2, 4 DA1 268 GiB normal ok
e3d3s04 2, 4 DA1 272 GiB normal ok
e3d3s05 2, 4 DA1 272 GiB normal ok
e3d3s06 2, 4 DA1 272 GiB normal ok
e3d4s01 2, 4 DA1 272 GiB normal ok
e3d4s02 2, 4 DA1 272 GiB normal ok
e3d4s03 2, 4 DA1 272 GiB normal ok
e3d4s04 2, 4 DA1 272 GiB normal ok
e3d4s05 2, 4 DA1 272 GiB normal ok
e3d4s06 2, 4 DA1 272 GiB normal ok
e3d5s01 2, 4 DA1 272 GiB normal ok
e3d5s02 2, 4 DA1 272 GiB normal ok
e3d5s03 2, 4 DA1 272 GiB normal ok
e3d5s04 2, 4 DA1 272 GiB normal ok
e3d5s05 2, 4 DA1 272 GiB normal ok
e3d5s06 2, 4 DA1 272 GiB normal ok
e4d1s01 2, 4 DA1 276 GiB normal ok
e4d1s02 2, 4 DA1 276 GiB normal ok
e4d1s04 2, 4 DA1 272 GiB normal ok
e4d1s05 2, 4 DA1 276 GiB normal ok
e4d1s06 2, 4 DA1 272 GiB normal ok
e4d2s01 2, 4 DA1 272 GiB normal ok
e4d2s02 2, 4 DA1 272 GiB normal ok
10
