HPE Storage Users Group
https://3parug.net/

3par 7200 double node failure after moving to new rack
https://3parug.net/viewtopic.php?f=18&t=3307		 Page 1 of 1
Author:  keenerb [ Tue Nov 12, 2019 11:49 am ]
Post subject:  3par 7200 double node failure after moving to new rack
I re-racked a 3par 7200 we had shipped up from our DR datacenter to use for test/training environment.

All data cables match 3par documentation and are configured the same as pre-unracking DR config, documented with notes and photos.

Unit powers on, amber LED illuminates and no unit identifier number appears on the front. Both power supplies have BATTERY FAILURE led illuminated steady, not blinking. Serial console reports nothing.

Controllers themselves illuminate all three status LEDs, then HOT PLUG and NODE FAULT begin flashing. After a few seconds, controller 0 stops flashing for a litlte bit and then starts flashing again.

Any suggestions? Could it be two simultaneous failed batteries?
Author:  MammaGutt [ Tue Nov 12, 2019 12:11 pm ]
Post subject:  Re: 3par 7200 double node failure after moving to new rack
If serial console reports nothing on either nodes, I would almost guess both nodes had failed.

Did you reboot nodes or power cycle the system before moving it to ensure there wasn't any latent hardware failures in the system?
Author:  keenerb [ Tue Nov 12, 2019 12:15 pm ]
Post subject:  Re: 3par 7200 double node failure after moving to new rack
Nope. I verified there was nothing reported in the management console, that's all.

I don't care about the data on the unit; is it possible to recovery from a double node failure without HP direct involvement?
Author:  MammaGutt [ Tue Nov 12, 2019 12:40 pm ]
Post subject:  Re: 3par 7200 double node failure after moving to new rack
If the nodes have failed, you need new nodes.

If my memory serves me right, a replaced node gets the system details from the remaining node. If both have failed, you have none working so there is no one to get the information from and you would have to get the information manually entered in the nodes. Secondly you would have to get the 3PAR OS loaded on the nodes, however this is also usually done from one of the other working nodes in the system.

So nothing is impossible, but I have no idea where to start :)

Edit: friendly advise for the future. Always test the hardware before moving anything, at least older storage systems. Reboot nodes one by one, and then full power cycle by flipping the circut breakers.

By doing that you can fix issues upfront and you also have proof towards transportation company if something breaks on the way from A to B.
Author:  keenerb [ Tue Nov 12, 2019 12:44 pm ]
Post subject:  Re: 3par 7200 double node failure after moving to new rack
That mirrors my line of thinking.

We shall see how this goes. We have third-party maintenance from a local company, I'll see what they have to say.

I think it's possible to failed PSU batteries might prevent system from booting, in order to protect data. I can't find any reliable information on that though.
Author:  MammaGutt [ Wed Nov 13, 2019 4:05 am ]
Post subject:  Re: 3par 7200 double node failure after moving to new rack
keenerb wrote:
That mirrors my line of thinking.

We shall see how this goes. We have third-party maintenance from a local company, I'll see what they have to say.

I think it's possible to failed PSU batteries might prevent system from booting, in order to protect data. I can't find any reliable information on that though.


My assumption is that it would disable write-cache.

But, there was a patch released not that long ago that allowed 2-node systems to boot with a failed node. However, it you had one failed node, the other one should have had serial output.
Author:  feldsam [ Wed Nov 13, 2019 5:03 pm ]
Post subject:  Re: 3par 7200 double node failure after moving to new rack
If boot drives are ok, then you can get two working nodes and just swap boot drive ssd into it
Author:  keenerb [ Tue Nov 19, 2019 9:30 am ]
Post subject:  Re: 3par 7200 double node failure after moving to new rack
feldsam wrote:
If boot drives are ok, then you can get two working nodes and just swap boot drive ssd into it


I had considered this as well. Do you know if these are standard SATA drives? Can I inspect them with a SATA to USB adapter? What filesystem are they?
Author:  MammaGutt [ Tue Nov 19, 2019 10:17 am ]
Post subject:  Re: 3par 7200 double node failure after moving to new rack
Google says that this is one of the models used : https://www.ebay.com/itm/LOT-OF-10-SanD ... 3350774864

So looks like SATA connector.

I would guess it is some type of Linux filesystem, but I would also expect it to be encrypted or something.
Page 1 of 1 All times are UTC - 5 hours
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/