Upgrade to 3.1.3 pre planning gotchas

[woodunn]

We have 5 arrays in our environment
2 x T800 3.1.2 MU1
2 x V800 3.1.2 MU2
1 X F400 3.1.2 MU1

We are going through the pre planning for 3.1.3 as it looks to fix about 5 open calls we have with HP.
We are coming across a few gotchas in the persona changes required.
Persona 6 can no longer be used after 3.1.3 and to change it to 11 ALL hosts in the ESX host set need to be shut down at the same time to change this.
Our beef with this is our ESX environment consist of 625, 481 and 161 production guests respectively and to change the persona would mean a complete outage!
To us this is not an enterprise solution. We will wait for HP's response.

Has anyone come across any other gotcha's?

[Richard Siemers]

The pre-planning upgrade doc seems to be a long list of dirty laundry.

So the document I was looking at is here:
http://h20565.www2.hp.com/portal/site/h ... cachetoken

Here's what I found in the docs:
Host Persona 6 will not be supported for any version of VMware ESX/ESXi with future HP 3PAR OS versions after HP 3PAR OS 3.1.3 and its MUs. HP recommends that customers migrate their VMware ESX configurations on HP 3PAR to Host Persona 11 with HP 3PAR OS 3.1.2 or later. HP recommends host persona 11 for all supported VMware ESX versions with HP 3PAR OS 3.1.2 and later. Changing the persona from 6 to 11 for existing Persona 6 hosts is an offline process.

I ideally we would just do that 1 host at a time but you said that "ALL hosts in the ESX host set need to be shut down at the same time to change this", can you point me in the direction of where that additional "All Simultaneously" requirement is documented?

Thanks!

[woodunn]

Thanks Richard,
As the document did not state how/if the change to an ESX's host persona to 11 would have an affect of another ESX host (which is still using persona 6) access to a shared VLUN.

We have asked HP via official channels to clairfy the effect on changing one ESX host at a time in a host set. We have been getting either cut and past from the document or a non committal "I think it should be ok BUT.....)

One of our engineers is waiting on a response to the following.


At present all our ESX hosts talk to the array using persona 6 (Generic-legacy – no persona capabilities) and on each array they see the same LUNs via a host set.

As soon as we change one of the ESX hosts to persona 11 (VMware – SubLun, ALUA persona capabilities) and boot it up, it will begin talking to the virtual volumes via a different set of rules (due to the changed persona capabilities) while the remaining ESX hosts continue to use the old ones.

My concern is that the different communication rules will cause ripple effects in VMware. We used to have this on an old array where hosts were changing the preferred path causing a seesawing effect until VMs begin crashing. We don’t want that again.


We are waiting on a response.

[apol]

Hi,

on our two V400, we "storage-guys" provided a script for every esx-server for usw by the ESX-guys to change the Host Persona when they did their last ESX-Update. As they did the updates cluster-wise (they didn't want to use different esx-versions in a cluster at the same time for too long), the longest period of time ESX with different personas accessed the same volumes was mere hours or some days. We did not encounter any problems.

IMHO, Persona 11 is just a hint for the array as to what expect from a host, as sublun-adressing is part of the vaai-primitives (persona 6 or 11), and alua only kicks in if you have the secondaries exported as well - what you hopefully haven't with perona 6 - hosts.

What we did encounter with the 3.1.3 update on our test arrays was a problem with the new path_management-policy. Simplified, this policy tells the array if it should enable alua on a volume/remote copy group or not. With secondaries exported (for peer persistence), you need path_management set, but the default is no_path_management. Apparently, the installer uses the state of the auto_failover-policy to determine for which remote copy group to enable path_management. The problem: We currently use the manual transparent failover option with no auto_failover-policy or quorum in use. This means we need path_management set, but didn't get it --> our ESX-Lab went down HARD. I think this is an error in the installer, it seems HP just forgot the possibility that somebody licensed and uses Peer Persistence without the automatism/quorum device.

Solution: Set the auto_failover-policy for the update for all remote copy groups where the secondaries get exportet to the esx hosts for manual transparent failover. Unset it after the update.

[mugurs]

Hi,

On one of our V800 array we have a similar case as some of our ESX hosts are registered with Persona 1 and other ESX hosts with Persona 11 and they all have been given the same LUNs. For the hosts with Persona 1 we plan to shut them down one by one to modify Persona to 11.

mugurs

[hdtvguy]

We have 5 arrays in our environment
2 x T800 3.1.2 MU1
2 x V800 3.1.2 MU2
1 X F400 3.1.2 MU1

We are going through the pre planning for 3.1.3 as it looks to fix about 5 open calls we have with HP.
We are coming across a few gotchas in the persona changes required.
Persona 6 can no longer be used after 3.1.3 and to change it to 11 ALL hosts in the ESX host set need to be shut down at the same time to change this.
Our beef with this is our ESX environment consist of 625, 481 and 161 production guests respectively and to change the persona would mean a complete outage!
To us this is not an enterprise solution. We will wait for HP's response.

Has anyone come across any other gotcha's?

I agree. We have been caught by numerous short-sighted planning by HP for a supposed "Enterprise" array. Another thing if you have Qlogic HBAs check your firmware. We had a bug with Dell blades with Qlogic HBAs that the HBAs did not like persona 11 and wound up hanging during POST. I do think having to power off (for us 130+ blades) because of the Persona change was unacceptable.

Why is it a complete outage for you? How many hosts do you have? We did a rolling upgrade, we did Qlogic firmware, powered off blade, changed persona and powered back up. We dia few blades at a time. Granted it sucked and took large amount of resources vs. an automated upgrade, but we got it done without any outages.

My account team does not agree, but 3par's niche roots still come glaring through as they struggle to think Enterprise.


e my other posts as well, make sure any RCIP and mgmt ports are set to AUTO or your upgrade will fail.

[Cleanur]

Just to put a bit of perspective around this, you have between now and the end of life of the new 3.1.3 release to make the change to host persona 11, so there's time to plan for this.

The change is needed to support additional features coming down the pipe from both HP & VMware and will avoid confusion around multiple persona's and the continued need to test and maintain the same across a single hypervisor going forward.

Taken from the release notes

HP 3PAR OS 3.1.3 supports Host Persona 6 and 11 for VMware vSphere host Operating Systems. HP 3PAR OS 3.1.3 is the last major HP 3PAR OS release to support Host Persona 6. Subsequent HP 3PAR OS versions will exclusively use Host Persona 11 for all supported ESX versions.
................................
HP 3PAR OS 3.1.3 supports Host Persona 15 for Windows 2008, Windows 2008 R2, and Windows 2012. Hosts with these operating systems must be converted to Host Persona 15 from Host Persona 2 after the array is upgraded to HP 3PAR OS 3.1.3. This can be completed with the host online and active.

The change can't be achieved online for an ESX hosts, but given the same can be for Windows hosts that would point to a possible limitation within the ESX stack.

[afidel]

Just to put a bit of perspective around this, you have between now and the end of life of the new 3.1.3 release to make the change to host persona 11, so there's time to plan for this.

The change is needed to support additional features coming down the pipe from both HP & VMware and will avoid confusion around multiple persona's and the continued need to test and maintain the same across a single hypervisor going forward.

Taken from the release notes

HP 3PAR OS 3.1.3 supports Host Persona 6 and 11 for VMware vSphere host Operating Systems. HP 3PAR OS 3.1.3 is the last major HP 3PAR OS release to support Host Persona 6. Subsequent HP 3PAR OS versions will exclusively use Host Persona 11 for all supported ESX versions.
................................
HP 3PAR OS 3.1.3 supports Host Persona 15 for Windows 2008, Windows 2008 R2, and Windows 2012. Hosts with these operating systems must be converted to Host Persona 15 from Host Persona 2 after the array is upgraded to HP 3PAR OS 3.1.3. This can be completed with the host online and active.

The change can't be achieved online for an ESX hosts, but given the same can be for Windows hosts that would point to a possible limitation within the ESX stack.

The Windows storage stack is more robust and forgiving in general. If I had one main complaint about VMWare it's that their storage stack is WAY too fragile in the face of non-optimal or unexpected storage events. We've had way more disruption caused by storage behavior on VMWare than all other causes combined which is certainly not the case with Windows or even Xen.

[hdtvguy]

The Windows storage stack is more robust and forgiving in general. If I had one main complaint about VMWare it's that their storage stack is WAY too fragile in the face of non-optimal or unexpected storage events. We've had way more disruption caused by storage behavior on VMWare than all other causes combined which is certainly not the case with Windows or even Xen.

Agreed! And not to get off topis; what is worse is vmware does not consider the loss of storage to be a HA fail over even. When we had an blade issue that caused the HBA to loose its WWNs the vmware host basically lost all attached storage for all the guests and it just keep going on, never failed over. Escalated to vmware and their answer is behaving as designed, if storage is totally gone but the management nic responds then the host does not consider that to be a HA event. What a stupid design. I am no Hyper-V fan, but love that they are creating pressure for vmware to step up their game.

[Richard Siemers]

VMware storage issues... double check your VMFS versions. One issue we ran into twice now is a primitive unreleased SCSI lock on a datastore LUN... we've seen this cause hosts to boot extremely slow, and complete mayhem under vSphere since only 1 host in the cluster is allowed to see the LUN.

New datastores running VMFS-5 (not old ones upgraded to 5) should remove the freak orphaned scsi-3 lock issue.