Failed Cage? after insert replacement HDD

[KevinT]

Hi all,
Hope all still well.

Physically replaced a failed HDD on HP_3PAR 8400, before logically ejecting/removing/whichever command needed to be executed.

Disk shows below info, there are 2 entries for cage 1 7 (15 & 16)

15 1:7:0 normal 5000C500975934C0 SEAGATE STHB1200S5xeN010 S401FP0Q 3P02 SAS Magnetic 2021-03-18 15:19:04 SAST
16 1:7:0? degraded 5000CCA0320BE72B HGST HCBF1200S5xeN010 06V6JXXZ 3P03 SAS Magnetic 2021-03-11 12:43:27 SAST
------------------------------------------------------------------------------------------------------
Servicemag resume shows it is still busy or been interrupted, any way to recover from this?

Cage 1 mag 7 'servicemag resume' was started since Thu Mar 18 17:40:08 2021 or it has been interrupted. Please run 'servicemag status -d' for further details
servicemag resume -partial 1 7 -- Failed
Command failed

Thanks in advance.

[MammaGutt]

The question mark bare 1:7:0 on PD16 indicates that the drive is missing. That is usually a sign of servicemag not completing and that there is most likely some chunklets stuck on the "ghost" PD ID 16 that needs to be cleared before you could dismiss the PD ID.


"showpd -c -cg 1 -pn 7" should provide verification.

"showpdch -mov" _could_ possibly show you the chunklets in question

[KevinT]

Hi,
thanks for the quick response, much appreciated.

syntax error?

showpd -c -cg 1 -pn 7
showpd: Invalid option: -cg

amended:
showpdch -mov shows normal/valid for all

[KevinT]

this syntax ok?

showpd -c 1 7
------- Normal Chunklets -------- ---- Spare Chunklets ----
- Used - -------- Unused -------- - Used - ---- Unused ----
Id CagePos Type State Total OK Fail Free Uninit Unavail Fail OK Fail Free Uninit Fail
1 0:1:0 FC normal 1116 377 0 599 0 0 0 2 0 138 0 0
7 0:7:0 FC normal 1116 372 0 604 0 0 0 6 0 134 0 0
----------------------------------------------------------------------------------------
2 total 2232 749 0 1203 0 0 0 8 0 272 0 0

[KevinT]

I also received this mail after running admithw in case it can help.

Event urgency: alert
Event count: 1
Event location: Site
Event time: 2021/05/26 11:02:56.00 (-0700 PDT)
Event description: 3PAR INSERV evt_pdata_noconf_notstarted

Abstract:
(Minor)Preserved data LDs have not been started up.(evt_pdata_noconf_notstarted)

Text:
Event id: 43343732 Node 0 Cust Alert - Yes, Svc Alert - Yes Severity: Minor

Event time: Wed May 26 11:02:56 2021
Event type: evt_pdata_noconf_notstarted Alert ID: 5 Msg
ID: 280008
Component: Preserved Data
Short Dsc: Preserved data LDs have not been started up.
Event String: Preserved data LDs have not been started up.

[MammaGutt]

this syntax ok?

showpd -c 1 7
------- Normal Chunklets -------- ---- Spare Chunklets ----
- Used - -------- Unused -------- - Used - ---- Unused ----
Id CagePos Type State Total OK Fail Free Uninit Unavail Fail OK Fail Free Uninit Fail
1 0:1:0 FC normal 1116 377 0 599 0 0 0 2 0 138 0 0
7 0:7:0 FC normal 1116 372 0 604 0 0 0 6 0 134 0 0
----------------------------------------------------------------------------------------
2 total 2232 749 0 1203 0 0 0 8 0 272 0 0

showpd -c 15 16

[MammaGutt]

Hi,

amended:
showpdch -mov shows normal/valid for all

That output only shows chunklets that should be moved. So even if chunklets are listed "normal/valid", any chunklet listed are "in the wrong place".

[KevinT]

Hi there,
See output of showpd.

showpd -c 15 16

------- Normal Chunklets -------- ---- Spare Chunklets ----
- Used - -------- Unused -------- - Used - ---- Unused ----
Id CagePos Type State Total OK Fail Free Uninit Unavail Fail OK Fail Free Uninit Fail
15 1:7:0 FC normal 1116 0 0 260 856 0 0 0 0 0 0 0
16 1:7:0? FC degraded 1116 0 0 624 353 0 0 0 0 139 0 0
------------------------------------------------------------------------------------------
2 total 2232 0 0 884 1209 0 0 0 0 139 0 0

[MammaGutt]

Hi there,
See output of showpd.

showpd -c 15 16

------- Normal Chunklets -------- ---- Spare Chunklets ----
- Used - -------- Unused -------- - Used - ---- Unused ----
Id CagePos Type State Total OK Fail Free Uninit Unavail Fail OK Fail Free Uninit Fail
15 1:7:0 FC normal 1116 0 0 260 856 0 0 0 0 0 0 0
16 1:7:0? FC degraded 1116 0 0 624 353 0 0 0 0 139 0 0
------------------------------------------------------------------------------------------
2 total 2232 0 0 884 1209 0 0 0 0 139 0 0

So the missing drive has a total of 1116. Of those, 624 are listed as "Free", 353 is listed as "Uninit" and 139 is listed as "unused spare". So nothing bad there.

That brings me back to the showpdch -mov command ... and it could be as simple as removespare and dismiss the pd.

[KevinT]

edited: do you need the complete output of showpdch -mov?


Hi there,
Thanks.

dismisspd 16
Error : Pd id 16 is referenced by chunklet 0:938

movepd 16
There are no chunklets to move.

Strange thing is, there are only 16 physical PD in the array, but showpd shows 17 devices?
showpd
It's like 15 & 16 are the same disk.

----Size(MB)----- ----Ports----
Id CagePos Type RPM State Total Free A B Capacity(GB)
0 0:0:0 FC 10 normal 1142784 754688 0:1:1* 1:1:1 1200
1 0:1:0 FC 10 normal 1142784 754688 0:1:1 1:1:1* 1200
2 0:2:0 FC 10 normal 1142784 755712 0:1:1* 1:1:1 1200
3 0:3:0 FC 10 normal 1142784 755712 0:1:1 1:1:1* 1200
4 0:4:0 FC 10 normal 1142784 755712 0:1:1* 1:1:1 1200
5 0:5:0 FC 10 normal 1142784 755712 0:1:1 1:1:1* 1200
6 0:6:0 FC 10 normal 1142784 755712 0:1:1* 1:1:1 1200
7 0:7:0 FC 10 normal 1142784 755712 0:1:1 1:1:1* 1200
8 1:0:0 FC 10 normal 1142784 684032 2:1:1* 3:1:1 1200
9 1:1:0 FC 10 normal 1142784 770048 2:1:1 3:1:1* 1200
10 1:2:0 FC 10 normal 1142784 679936 2:1:1* 3:1:1 1200
11 1:3:0 FC 10 normal 1142784 771072 2:1:1 3:1:1* 1200
12 1:4:0 FC 10 normal 1142784 681984 2:1:1* 3:1:1 1200
13 1:5:0 FC 10 normal 1142784 770048 2:1:1 3:1:1* 1200
14 1:6:0 FC 10 normal 1142784 680960 2:1:1* 3:1:1 1200
15 1:7:0 FC 10 normal 1142784 1142784 2:1:1 3:1:1* 1200
16 1:7:0? FC 10 degraded 1142784 1142784 ----- ----- 1200
-------------------------------------------------------------------------
17 total 19427328 13367296