Help creating GPG - 7450 with SSD

[oester]

I'm trying to create a raid-5 CPG on my 7450. Two cages of SSD, 24 400GB drives each. I'm trying to use a command like:

createcpg -t r5 -ssz 4 -rs 6 -p -nd 0,1 -devtype SSD ssd-r5-ssz8-a

But I always get the error:

Error: no available space for given (invalid?) parameters

There are no CPG's defined at present. I've tried some other variations - always the same error.

Bob

[Richard Siemers]

So some of those settings may not be ideal unless you are a 3par ninja, and know exactly what you want, in which case you probably wouldn't be asking for help here... so I am going to assume you are not a ninja, and just a regular 3PAR user like many of us.

createcpg -t r5 -ssz 4 -rs 6 -p -nd 0,1 -devtype SSD ssd-r5-ssz8-a

I notice the -ha is missing. I believe the default "high availability" setting is CAGE... so in order to create a r5 ssz 4 you would need to have at least 4 disk shelves with SSD installed. You may be forced to use -ha mag to force this CPG to be created, but I would like give you some more info before you do that.

If you only have 2 nodes, you can leave out the -nd 0,1. If you have 4 (or more) nodes, you should discuss that with your HP professionals (or perhaps us here online) as it is best practice and best performance for your CPGs to span as many drives as possible. I would recommend you have 1 cpg for all of your SSD, 1 for all of your FC, and 1 cpg for all of your nearline. If your intention is to isolate "spindles" from each other as per old school legacy storage/database best practice, I recommend re-evaluating that plan and downloading some of the HP 3PAR implementation guides that discuss best practice for Exchange/SQL/ or Oracle specifically.

-rs 6 ; This one throws me off the most, what is the design/intent here here by manually setting the row size instead of letting the system decide that?

Raid 5 set size 4 on SSD. The SSD space is so expensive, I totally understand the desire for raid 5 to get 25% more usable space, I argued this case myself for my own site. However, raid 5 on ssd is counter productive to performance goals. The raid 5 write penalty combined with SSD re-write penalties spread across your set size, downgrade performance significantly. Not to mention the parity writes put a higher re-write load on the SSD and wear them out faster. Raid 1 for SSD is best option to deliver SSD performance for both reads and writes, plus if you only have 2 disk shelves, its easier to maintain cage availability.

If you could share you config, model number, number of nodes, number of shelves, number of SSD drives... and your goal, myself or someone else will be able to provide better information.

[oester]

Yea, it was the "-ha" option, it needed to be set to "mag" to do anything other than raid-1. My bad. And I was copying an old command from my 7400's -hence the wrong "rs" value. I did get it created - raid-5 (3+1)

However - interesting comments on raid-1 vs raid-5. My particular application is IBM GPFS - using the SSD for metadata. I have trouble swallowing the 50% loss of disk space for raid-1. Each of them (I have two 7450's) has 48-400 GB SSD's. Just looking to get the maximum performance and the leas amount of lost disk I can and still get protection.

I had carved up my 7400's with two raid-5 CPG's - one on nodes 0,1 and one on node 2,3 - so I was looking at doing something similar here - but I may be way off base.

Suggestions welcome on how to best configure the CPG's on the 7450's with SSD.

Bob

[Richard Siemers]

Yeah I had the same trouble swallowing the 50% overhead for raid 1 on SSD. My SE tried to talk me out of it, but what pushed me to Raid 1 was after I googled "raid 5 on SSD".

I recommend not splitting your CPGs by controller pairs and letting them span as many spindles across as many nodes as possible to maximize the benefit of wide striping.

[oester]

Well - but are we really comparing apples-apples here? With the 3par design, spreading chunklets across multiple drives a 20GB raid-5 array is really spread across 24 SSDs, with 4 of them being used for parity. I'm not sure the research really holds up in this environment. Of course I could be wrong - lots of new SSD specific algorithms out there, but it will be a while before we see any vendor support.

[Architect]

Why are you confining the CPGs to one nodepair only? this could impact your backend performance, depending how your hosts are connected. Any deeper thoughts on that design you'd like to share? maybe i'm overseeing something.

For 99.99% of the clients i come across R5 3+1 is the best solution for SSD. why? because SSD can handle way more performance per GB then the avg. application can throw at it. Usually the SSD is full (capacity wise) before the performance limit is reached. This even with R5 3+1.

What is the amount of iops you are expecting to do (read/write)? and how large is the database you are writing to SSD? just curious to see what special needs you have.

[Cleanur]

Agree Raid 5 3+1 is best practice on SSD these days unless you have a very very heavy write workload.

You shouldn't prefer CPG's to node pairs unless again you have a very specific goal in mind and are willing to micro manage that configuration going forward. Best practice would be to have a single CPG spanning all nodes, other wise your effectively undermining 3PAR's ease of use, efficiency and Symmetric Active-Active architecture and making it look more like a legacy active-passive array.

Just because you can do something doesn't necessarily mean you should.

[Architect]

Well, we could start an interesting discussion on local vs remote IO and the microseconds of delay the fetch from remote node would cause But fully agreed on that a 3PAR is a wide striping architecture and it in general performs best when it does what it can do what it is designed for: wide stripe all load.