CPG / VV Advice

[slink]

I'm a total noob to storage and SANs and pretty new to Hyper-V and 2012 but I've been tasked with setting up a 2012 Hyper-V cluster on a 3PAR 7200 with 24 x 900GB 10K SAS drives in it. I'm about to wade through the Storage Concepts guide and the other material but I just wanted to throw a post out there and maybe get some insights while I'm reading and learning.

How should I go about provisioning this storage for Hyper-V? It's going to hold VMs and for all the common Microsoft 2012/13 services - Exchange, SharePoint, SQL 2012, Lync etc. It's a fairly small environment, there'll only be a couple of Exchange VMs, 2 Lync VMs, 3 or 4 SharePoint and on like that. Only 1000 users using it all. One thing I do want to setup is guest clustering with SQL 2012 so I'd need to separate some shared storage for that.

Do I create one big RAID1 CPG using all 24 disks and then two TPVVs to present one for use as the CSV for Hyper-V and one for use by the SQL cluster? Or should I be creating smaller TPVVs and using multiple CSVs? I'm not sure whether I should be chopping up this storage all at once or provisioning it as and when. I've read that less CSVs are better so I thought I should just do one big one.

As you can probably tell, I have no idea and have some learning to do but any advice you can give me would help, I've got a week to get this up and running

[jeffgray]

Hello,

I'm about to do just about the same thing you're doing. Installation is next week. I'll have 48 15K drives and 8 SSD in a 7200.

My plan is to use SystemCenter VMM which will allocate storage for each VM on the fly. I think. In theory, I'll be able to map SCVMM storage service levels to CPGs of varying performance levels in the 3PAR.

I've been experimenting with SQL clustering in HV 2012 on Lefthand storage and I know you will want several LUNs for that since CSV does not work for the shared storage in guest clustering. I can't test the SCVMM integration as I don't have the SMI-S capable Lefthand SAN IQ version yet.


I'll let you know what I figure out. Good Luck! Hopefully we'll get some solid guru mojo here...

[slink]

Cool, whatever you decide to do let me know. From reading the concepts guide I'm wondering whether I should bother with Thin Provisioning on the VVs at all. Seems like there's an added risk there or at least, a greater amount of monitoring and management required to make sure everything is OK. Since I'm not going to be limited for storage to begin with I'm leaning towards creating a 5TB fixed VV for the CSV I can put all the VMs on and another 2TB fixed VV for the SQL cluster. Since all my disks are the same I don't really need to tier them and I'm happy to go with RAID1 for everything as the performance will be better. Starting off with 7TB allocated gives me some headroom for growth, snapshots etc. and it also seems fairly easy to move from fixed to thin with the 3PAR if I found I needed to shuffle stuff around.

Why would I need several LUNs for teh SQL? With all this storage virtualisation and abstraction is there any real need to separate out Logs and Data onto separate LUNs since everything is just being striped across the same chunklets I'm thinking multiple LUNs just adds complexity without any performance gain.

What about CSVs? How are you planning to structure those?

[jeffgray]

For clustering, you might find that you need a witness disk (aka quorum disk) and one for DTC if you decide to cluster that.
I've always created separate LUNs for data and log volumes despite the fact that they're allocated from the same resources. There are two reasons for me: One is that I can provision/change performance characteristics which may be benficial to specific workload types. Putting everything on one LUN requires me to move files around if I want to change the provisioning. It may not be applicable in your case since you plan on using the same raid level across all of your LUNs. OK VVs. I'm still getting used to the lingo.

The other reason for separation is recovery. If for some reason Windows makes a mistake that results in a corrupted filesystem and you lose your SQL data files, you can perform a "tail" log backup and recover all of your data. If that happens and they're both on the same filesystem, you are stuck with recovery back to the point of your last transaction log backup. It's a few minutes of extra work to set up, and it's never helped, but I do it anyway.

I think that I will avoid using CSVs in my implementation, as SCVMM will provision disks for each VM by direct communication to the array. Again I only think that's how it will work as I've not been able to play.

If I do end up using CSVs, I will at least create two, and avoid putting the non-shared disk resources of the node pairs on the same CSV. With Lefthand, I've created lots of volumes with a few VMs on each. With the zero reclamation in 3PAR I don't think I would do that. (And now the T10 unmap shows up in SAN IQ)

[slink]

how will your VMs be highly available if you don't use any CSVs?

Also regarding the separation of SQL, agreed I will need a separate VV for this as I can't use a CSV but why do I have to separate Logs and Data at the LUN level? You could create separate VHD's (my intention) or even just separate logical partitions in a single VHD?

I agree that in a traditional SAN I would put these on separate LUNs but in the 3PAR world on an all RAID1 CPG, is there any need?

[slink]

OK I've got everything up and running and am in a position to create my first CPG. Anybody want to chime in on how you provision your storage to give me some ideas on how best to approach this? Fixed vs Thin? Several CPGs and VVs or just a few?

My current thinking is to do 2 x CPGs, both of RAID1 using cage availability and half the disks in each.

Then I would have:

CPG1

VV1: CSV1

VV2: SQLDB

CPG2

VV1: CSV2

VV2: SQLLOG

So I'm separating out the Read/Write workloads and also giving myself 2 CSVs on which to balance out pairs of VMs for better HA. However, as I mentioned above with my query about data/log separation, with these both being RAID1 CPGs and the 3PAR doing all this clever chunklet stuff, do I need 2 CPGs? How about using all the spindles in 1 CPG and putting these 4 VVs on that? Bad idea?

[afidel]

I wouldn't specify individual disks, that's really fighting the design and purpose of the 3Par. Also, have you seen the 3Par paper on Oracle performance? They showed performance for RAID5 3+1 that was only 9% lower than RAID10 in an OLTP setup, for me getting 50% more space was a lot more important than 9% performance. I'd rather throw a couple SSD's at the hot blocks than waste a bunch of disk capacity.

[slink]

thanks for the reply

yeah, this is what I don't quite understand, I didn't think splitting up the disks would be a good idea so then what's the point in having multiple CPG's if they're going across all the disks anyway? If I'm planning to use RAID1 (which I probably will) then I should just have one big CPG across the whole lot fright? Then just have my 4 VV's sitting on the one CPG.

[afidel]

You can set different policies per CPG, and a CPG can only be in one AO policy. So I have about a dozen different CPG's, some have RAID5 7+1 for capacity, some have RAID5 3+1 for performance, some are in AO policies that allow movement between all 3 tiers of disk and some are in AO policies that limit how far up or down blocks can move, and finally a few aren't in an AO policy because I want to pin that data to a specific tier (Exchange archive database is always on NL, nonprod database servers are always on FC, I don't want to waste precious SSD space on nonprod but I don't want performance to suck either).

As to your specific case, yes you can do just one CPG with RAID10. However I'd personally put them on RAID5 3+1 and then only if performance seems to be a problem do a DO move to RAID10 (assuming you have the DO license).

[slink]

I have the DO license but not an AO license. I also have a shelf of 2TB NL disks but that is going to be for backup and archive so I was going to create a separate CPG for all that.

Maybe I'll create a RAID3+1 then for the CSV volumes and SQL and Exchange Data and a RAID10 for the log volumes.