3par Support?

[Lorax]

I'll throw my hat in the ring here...

We've had a pretty mixed experience with HP support. We were a 3PAR customer just before HP bought them, and back then the support was exceptional - through to a knowledgeable U.S based engineer at the drop of a hat.

Dealing with HP support these days is fine for simple issues like drive replacement, but for more complex issues, it can be like pulling teeth. There's sometimes a language barrier, and the support engineers are not always as knowledgeable as their U.S counterparts. We had a massive outage which was caused by a HP 3PAR first line engineer earlier this year, which was really inexcusable.

We had a power problem in one of our datacenters, which resulted in us having to shut down our F200 before UPS power ran out. The InServ was down for more than 24 hours, which means that when it comes back up, you have to manually issue a TOC update. We had a HP engineer connected in to check things over, and he did this - but issued a TOC update from 5 days previous instead of the most recent one. To complicate things further, in between that TOC from 5 days previous, we'd had two drives replaced. So when the TOC was applied, there was a discrepancy between the drives the TOC was expecting, and the two new drives that it didn't know about... Long story short, 80% of the virtual volumes could not be brought back online, and in the end the InServ had to be zeroed out, and the InServ OS installed from scratch. Apparently as a result of this, the ability to issue this command was removed from first line support and only made available to second line.

[hdtvguy]

The complexity of the array is overwhelming for HP's support staff. Sure you need some of the complexity, but I attribute a lot of this to the pre-HP days when this was a niche product and only a handful of people were needed to support it. 3par NEVER thought about scaling this technology to large enterprises form a technology (RC sucks) or support (every time field support shows up they need to call back end support for something). We have about 18 months before we do another technology refresh and can say at this pace I will have a hard time picking 3par for future storage needs. I was a reference customer until 3-4 months ago when they kept screwing up everything they touched. Now I can not be a reference because even day to day stuff is getting painful. Deal with the IMC on an array that has 900+ exported volumes and over 5500 volumes (including snaps) and the IMC is unusable. And to think they are supporting 10s of thousands of volumes, really? Good luck managing that mess.

[Richard Siemers]

Vendor criticisms on the internet are a touchy subject. You won't see the vendor getting online to refute them because thats bad PR, or can violate a customer's privacy... it up to the reader to pick apart the details, or realize what details are missing, and come to their own conclusions.

Lorax - Sorry to hear about your disaster. How was the F200 powered off? What commands were used? Needing to apply a "TOC update" sounds bad, I am inclined to think that is only required if the unit was not cleanly shut down.

Kieth - I don't think its fair or accurate to say "3PAR never thought about scaling this technology to large enterprises". I think the 3PAR support model worked well, and had a plan and means for scaling. When HP bought them, that plan had to be redesigned to use HP resources, which caused alot of change, and introduced a lot of people unfamiliar with the product at the exact same time the product popularity was booming... double whamee.

In any high risk support situation with any vendor, I will bench an engineer/technician and escalate to someone else if I do not feel his competency level is aligned with the task at hand OR if their ability to communicate with me is not perfect.

[Josh26]

You won't see the vendor getting online to refute them

I don't know, look at Symantec, who sacked most of their product developers to replace them with "Social Media Engineers" in an effort to make Backup Exec a "better product"

Regarding the specific issue above though, I know while I 7200 was pre-deployment, we shut it down for a week at a time just because we weren't using it. With a clean shutdown + power unplugged we never had a problem.

I can certainly echo the sentiments around front-line technicians bringing things offline though.

Edit: If you check the 3.1.2 InForm OS release notes, you can see this one:

In HP 3PAR OS version 3.1.1, a check was added when selecting a TOC after system reboot to prevent automatically selecting a TOC more than a day old. this change is reverted.

It would appear this was a "feature" of 3.1.1, rolled back when someone realised how terrible it was.

[cookie]

Feel your pain, we came from another vendor and the software suites on 3par are very basic.

Tools seem to be missing to do things with any ease, hopefully PowerShell stuff will come along so we can at least script stuff.

Support on the RC suites goes round in circles and doesn't seem to move forward.

[hdtvguy]

Kieth - I don't think its fair or accurate to say "3PAR never thought about scaling this technology to large enterprises". I think the 3PAR support model worked well, and had a plan and means for scaling. When HP bought them, that plan had to be redesigned to use HP resources, which caused alot of change, and introduced a lot of people unfamiliar with the product at the exact same time the product popularity was booming... double whamee.

I am not sure I agree. They may have thought about it and had a plan, but they did not vet it out. When you look at their supported limits their corresponding tools and technology do not scale to meet those limits. Also their support model is very back end remote tech heavy and that model does not scale well for high volume products regardless if HP bought them or not, if their sales were at current levels they would have collapsed if not for HP's resources.

There is no way the current Remote Copy limitations work in a large array that has frequent short interval replications when you can only replicate 20 volumes at a time and there are known bugs when some of those volumes are multi-TB volumes that the array takes too long to process what changes occurred before it can even start the movement of data thus using up valuable slots of replication.

Also what Enterprise product does not have a concept of Consistency Groups for replication? There is no such thing in 3par RC Groups for periodic replication. And there is no current plan to fix this. Replication in 3par is a huge Achilles heel, very limited and does not scale up.

And don't get me started in the IMC, worst Java app I have ever used, has conflicts with many other apps, also can't handle hundred of volumes, let alone thousands. Last decent IMC was 4.2.0 all down hill from there.

They could have thought about scale-ability all day lone, but they did not architect it for it, even if they think they did.

[Richard Siemers]

Also what Enterprise product does not have a concept of Consistency Groups for replication? There is no such thing in 3par RC Groups for periodic replication.

Can you explain that further? Its my understanding that is exactly what a 3PAR RC copy groups are suppose to do. Attached is snippet from the documentation that supports that "understanding".

rc_group.JPG (82.32 KiB) Viewed 22721 times

[inex_77]

I experienced that EVA support was getting worse in 2007-2012 periode, but I have always been happy with my 3PAR support. I have also worked a lot with EMC storage products, and my experience is that EMC has good support, but not always the best products. For 3PAR is far superior to the VNX series.

[hdtvguy]

Also what Enterprise product does not have a concept of Consistency Groups for replication? There is no such thing in 3par RC Groups for periodic replication.

Can you explain that further? Its my understanding that is exactly what a 3PAR RC copy groups are suppose to do. Attached is snippet from the documentation that supports that "understanding".

rc_group.JPG

If you have multiple volumes in a group there is no way to make sure they are all consistent at the destination:

Let's say I have 4 volumes in a RC Group for a SQL server:

vv0 - boot drive
vv1 - data drive
vv2 - log drive
vv3 - tempdb drive

When RC starts it take a rcpy snap on the source side and destination side and begins to replicate the volumes from the state captured in the source side rcpy into the base volume at the destination.

Lets say vv0 and vv2 are done replicating, but vv1 and vv3 are still replicating and you have an event. Once the RC in process for vv1 and vv3 is interrupted the destination side will roll back those volumes to the rcpy of them on the destination side that represents them from the sate prior to the replication, but will leave vv0 and vv2 in the state they are in since they finished replicating before the event.

Two very important factors to consider here, first the 2 volumes that did not finish are rolling back to the state they were in before the replication and for a DB server in this scenario would result in an unusable server since the data drive is old and the log drive is current. Second you have to wait for those volumes to rollback before you can use those 2 since they are rolling back, in a large delta that could take numerous minutes and if you use the volume to take snap to use while that is occurring you are toast.

[Richard Siemers]

So 3PAR has a concept of consistency groups, and does take consistent group snaps of volumes (at the source) for periodic replication... your issue with it is that if the periodic replication update fails for whatever reason, it won't roll back the remote group to the last good copy. I misunderstood your original post, I thought you meant they had no consistency feature at all.

To your point, if the customer broke away from using the built in replication scheduler, and wrote their own scripts, could they not mitigate that issue by taking a snapshot of the remote replica before starting the group periodic update? Then delete the snap after successful replication?