HPE Storage Users Group https://3parug.net/ |
|
Update failed 3par OS 3.2.1MU5 to 3.2.2MU6 https://3parug.net/viewtopic.php?f=18&t=3575 |
Page 1 of 2 |
Author: | gnomba [ Tue May 18, 2021 11:17 am ] |
Post subject: | Update failed 3par OS 3.2.1MU5 to 3.2.2MU6 |
Hi guys! Help me to solve problem with update. Previously I never used any data storage systems. I have 3par 7400 INFORM OS 3.2.1.426 (MU5) patch P49,P53,P55. I installed Physical SP-4.4.0.GA-142 (MU9) for update to INFORM OS 3.2.2.709 (MU6). First try: Code: No InSplore running Terminated InSplores No InSplore running Release version 3.2.1.426 (MU5) Patches P49,P53,P55 Component Name Version CLI Server 3.2.1.483 (P53) CLI Client 3.2.1.483 System Manager 3.2.1.488 (P55) Kernel 3.2.1.426 (MU5) TPD Kernel Code 3.2.1.426 (MU5) TPD Kernel Patch 3.2.1.488 (P55) Selected upgrade type simple MESSAGE START State, No HPE 3PAR OS upgrade in progress STATE0 StoreServ release version 3.2.1.426 + P49,P53,P55 Beginning upgrade checks -- Test Name -------------------------------------------------------- RESULT - Check Defrag PASSED Check Local PASSED Check PD PASSED Check Upgrade PASSED Check Upgrade Offline PASSED Check VV PASSED Check Cluster Name Exists PASSED 3par001 is in an acceptable state to perform a software upgrade No InSplore currently running Stopping spevent task for StoreServ 3par001 Stopping spcollect tasks for StoreServ 3par001 If running, stopping InSplore for StoreServ 3par001 Transferring software may be a long running operation. Do not abort the operation. Installing update 3.2.2.709-MU6 Validating 3.2.2.709-MU6 Enter Simple Upgrade state(0) auto doupdate OS-3.2.2.709-MU6 substituting package for staged substituting package for staged Beginning upgrade of system software. Extracting... -/-/-/-/-/-/.-/-/-/-/-/-/.-/-/- Applying new package... Calculating checksums... -/.-/ Performing pre-update checks... - Following PD(s) exceeds the chunklet failure threshold and needs corrective action 5,6,7,2 Update failed. System is not currently undergoing an online upgrade. System is not currently undergoing an online upgrade. First node to be rebooted is node 1 Which will be followed by node 0 The upgrade has failed. Please view the messages above and contact HPE Customer Support (Error Code 1) Password Processing Done Second try: Code: No InSplore running
Terminated InSplores No InSplore running Release version 3.2.1.426 (MU5) Patches P49,P53,P55 Component Name Version CLI Server 3.2.1.483 (P53) CLI Client 3.2.1.483 System Manager 3.2.1.488 (P55) Kernel 3.2.1.426 (MU5) TPD Kernel Code 3.2.1.426 (MU5) TPD Kernel Patch 3.2.1.488 (P55) Selected upgrade type simple MESSAGE START State, No HPE 3PAR OS upgrade in progress STATE0 StoreServ release version 3.2.1.426 + P49,P53,P55 Beginning upgrade checks -- Test Name -------------------------------------------------------- RESULT - Check Defrag PASSED Check Local PASSED Check PD PASSED Check Upgrade PASSED Check Upgrade Offline PASSED Check VV PASSED Check Cluster Name Exists PASSED 3par001 is in an acceptable state to perform a software upgrade No InSplore currently running Stopping spevent task for StoreServ 3par001 Stopping spcollect tasks for StoreServ 3par001 If running, stopping InSplore for StoreServ 3par001 Transferring software may be a long running operation. Do not abort the operation. Installing update 3.2.2.709-MU6 Validating 3.2.2.709-MU6 Enter Simple Upgrade state(0) auto doupdate OS-3.2.2.709-MU6 System is not currently undergoing an online upgrade. First node to be rebooted is node 1 Which will be followed by node 0 The upgrade has failed. Please view the messages above and contact HPE Customer Support (Error Code 1) Password Processing Done system-wide certificate already exists Beginning post upgrade checks HPE 3PAR OS version(altroot) = 3.2.1.292+P17,P18,P19,P23,P30,P50,P52, rc(0) -- Test Name -------------------------------------------------------- RESULT - Check Defrag PASSED Health Check PASSED Post upgrade checks are passed but software upgrade failed on 3par001 due to incompatible state Starting spcollect tasks for StoreServ 3par001 Starting spevent task for StoreServ 3par001 There was an issue upgrading your StoreServ. Please contact HPE Customer Service with the output from the above operation to resolve this problem |
Author: | koopa79 [ Wed May 19, 2021 3:39 am ] |
Post subject: | Re: Update failed 3par OS 3.2.1MU5 to 3.2.2MU6 |
what do you get from a checkhealth -svc -detail The first screenshot suggests you have 4 physical disks with issues. I'm assuming they're not actually failed disks? Might need to fix those disks first. maybe try a showpd -degraded -failed could also try a showpd -c and it will show you how many failed chunklets on all disks. |
Author: | gnomba [ Wed May 19, 2021 4:49 am ] |
Post subject: | Re: Update failed 3par OS 3.2.1MU5 to 3.2.2MU6 |
checkhealth -svc -detail: Code: Checking alert Checking ao Checking cabling Checking cage Checking cert Checking dar Checking date Checking file Checking fs Checking host Checking ld Checking license Checking network Checking node Checking pd Checking pdch Checking port Checking qos Checking rc Checking snmp Checking task Checking vlun Checking vv Checking sp Component -------------------Description-------------------- Qty Alert New alerts 2 Node Filesystems which are over 90% full 1 Node Filesystems which are over 80% full 1 Host Host ports not configured for virtual port support 1 License Licenses which have expired 1 pdch LDs with chunklets on a remote disk 14 RC Structure Incompatibility 1 vlun Hosts not connected to a port 2 Component ---Identifier--- -------------------------------------------------------------Description-------------------------------------------------------------- Alert sw_os An Update is Available Alert sw_os Update OS-3.2.2.709-MU6 has failed: Following PD(s) exceeds the chunklet failure threshold and needs corrective action: 5,6,7,2----- Node node:0 Filesystem /dev/sda7 mounted on "/common" is over 90% full Node node:1 Filesystem /dev/sda7 mounted on "/common" is over 80% full Host Port:1:1:1 Port WWN not found on FC Fabric attached to Port:0:1:1 License Peer Motion License has expired pdch ld:worker0.usr.3 LD has 2 remote chunklets pdch ld:worker0.usr.1 LD has 3 remote chunklets pdch ld:admin.usr.2 LD has 2 remote chunklets pdch ld:master0.usr.5 LD has 16 remote chunklets pdch ld:master1.usr.5 LD has 16 remote chunklets pdch ld:.srdata.usr.1 LD has 13 remote chunklets pdch ld:master2.usr.5 LD has 16 remote chunklets pdch ld:master0.usr.3 LD has 2 remote chunklets pdch ld:worker0.usr.5 LD has 16 remote chunklets pdch ld:master0.usr.1 LD has 3 remote chunklets pdch ld:master1.usr.1 LD has 4 remote chunklets pdch ld:master1.usr.3 LD has 2 remote chunklets pdch ld:master2.usr.3 LD has 2 remote chunklets pdch ld:master2.usr.1 LD has 4 remote chunklets RC Structure Remote Copy internal structure is incompatible. vlun host:master0 Host iscsi_name:iqn.1994-05.com.redhat:3443780cbb is not connected to a port vlun host:master0 Host iscsi_name:iqn.1994-05.com.redhat:bfbbd4d9d631 is not connected to a port showpd -degraded -failed: Code: No PDs listed showpd -c: Code: ------- Normal Chunklets -------- ---- Spare Chunklets ----
- Used - -------- Unused -------- - Used - ---- Unused ---- Id CagePos Type State Total OK Fail Free Uninit Unavail Fail OK Fail Free Uninit Fail 0 0:0:0 SSD normal 1787 120 0 1220 0 0 0 0 0 447 0 0 1 0:1:0 SSD normal 1787 159 0 1181 0 0 0 0 0 447 0 0 2 0:2:0 SSD normal 1787 0 0 1333 0 0 7 0 0 447 0 0 3 0:3:0 SSD normal 1787 131 0 1204 0 0 5 0 0 447 0 0 4 0:4:0 SSD normal 1787 120 0 1215 0 0 5 0 0 447 0 0 5 0:5:0 SSD normal 1787 131 0 1202 0 0 7 0 0 447 0 0 6 0:6:0 SSD normal 1787 119 0 1215 0 0 7 0 0 446 0 0 7 0:7:0 SSD normal 1787 167 0 1167 0 0 7 0 0 446 0 0 ---------------------------------------------------------------------------------------- 8 total 14296 947 0 9737 0 0 38 0 0 3574 0 0 |
Author: | gnomba [ Wed May 19, 2021 4:57 am ] |
Post subject: | Re: Update failed 3par OS 3.2.1MU5 to 3.2.2MU6 |
I tryed fix failed chunklets on disk 2 in the following way: 1. connected disk to server 2. reformatted to 512 block size 3. blkdiscard 4. reformatted to 520 block size 5. returned disk to 3par But this did not solve the problem with failed chunklets. |
Author: | koopa79 [ Wed May 19, 2021 7:54 am ] |
Post subject: | Re: Update failed 3par OS 3.2.1MU5 to 3.2.2MU6 |
so at least you can see the reason the upgrade failed is those PD's as per Alert sw_os Update OS-3.2.2.709-MU6 has failed: Following PD(s) exceeds the chunklet failure threshold and needs corrective action: 5,6,7,2----- Now I've never thought about taking a disk out of 3par and connecting it to a server and formatting it. But i know the 3pars have a database of failed disks, my worry is now you've removed it I think its failed and has now marked that disk as bad. Because in the last screen shot, i don't like the OK column shows 0 on disk 2. I'm assuming this 3par isn't under support anymore? and that's why you haven't asked HPE to swap the disk. run another showpd -c and see if the details on disk 2 have changed anymore? i don't know if it has to rescan the chunklets again. |
Author: | gnomba [ Wed May 19, 2021 9:56 am ] |
Post subject: | Re: Update failed 3par OS 3.2.1MU5 to 3.2.2MU6 |
Yes, unfortunately there is no support for this storage. Before disconnecting disk 2, I manually transferred all used chunklets, but after returning disk 2, they apparently did not come back automatically, so the value in the OK column is 0. `showpd -c`: Code: ------- Normal Chunklets -------- ---- Spare Chunklets ----
- Used - -------- Unused -------- - Used - ---- Unused ---- Id CagePos Type State Total OK Fail Free Uninit Unavail Fail OK Fail Free Uninit Fail 0 0:0:0 SSD normal 1787 120 0 1220 0 0 0 0 0 447 0 0 1 0:1:0 SSD normal 1787 159 0 1181 0 0 0 0 0 447 0 0 2 0:2:0 SSD normal 1787 0 0 1333 0 0 7 0 0 447 0 0 3 0:3:0 SSD normal 1787 131 0 1204 0 0 5 0 0 447 0 0 4 0:4:0 SSD normal 1787 120 0 1215 0 0 5 0 0 447 0 0 5 0:5:0 SSD normal 1787 131 0 1202 0 0 7 0 0 447 0 0 6 0:6:0 SSD normal 1787 119 0 1215 0 0 7 0 0 446 0 0 7 0:7:0 SSD normal 1787 167 0 1167 0 0 7 0 0 446 0 0 ---------------------------------------------------------------------------------------- 8 total 14296 947 0 9737 0 0 38 0 0 3574 0 0 |
Author: | gnomba [ Wed May 19, 2021 10:00 am ] |
Post subject: | Re: Update failed 3par OS 3.2.1MU5 to 3.2.2MU6 |
Maybe I can somehow indicate not to use these failed chunklets or raise this threshold, for example, to 10? |
Author: | gnomba [ Thu May 20, 2021 7:03 am ] |
Post subject: | Re: Update failed 3par OS 3.2.1MU5 to 3.2.2MU6 |
I recently found out that I can unmap failed chunklets and update 3parOS. Only it is not clear how to do it. |
Author: | MammaGutt [ Thu May 20, 2021 8:38 am ] |
Post subject: | Re: Update failed 3par OS 3.2.1MU5 to 3.2.2MU6 |
Are you using the upgrade tool? |
Author: | gnomba [ Thu May 20, 2021 9:16 am ] |
Post subject: | Re: Update failed 3par OS 3.2.1MU5 to 3.2.2MU6 |
I tried update via Service Processor (WebUI). |
Page 1 of 2 | All times are UTC - 5 hours |
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group http://www.phpbb.com/ |