I'm fairly new at tracking down disk performance issues, so please bear with me. I'm also new to System Reporter and how to effectively use it to track down bottlenecks, which is one of the main drivers for this question. I'm not looking for all the answers, but really a dialogue to help me improve my understanding of the 3PAR hardware and how to make effective decisions relating to it.
I have a VM running in a farm with both FC and NL datastores. Our FC PD count is 224 and our NL PD count is 192. These datastores are spread across all of their respective spindles.
When we storage vmotion the VM from NL disks to FC disks we notice improvements in performance. This isn't surprising, but at the same time, I'm trying to better understand what to look at to know when something should be moved from NL to FC.
Clearly, the spindle count, seek time, and raw IOPSability of the FC are going to be major factors in the performance improvement. But, what are metrics that would be useful to review to know whether the issue is with a limitation of the NL vs FC or some other factor. I want to get the most out of our NL, and I want to provide compelling stories as to why something should be promoted. I'm focusing on the 3PAR side of things for now, because everything else appears to be the same, other than the underlying storage.
I'm sorry for the basic questions, but trying to build a foundation to allow me to dig deeper into analyzing our storage world. Thank you for any thoughts.
performance troubleshooting/tuning between NL and FC
-
Richard Siemers
- Site Admin
- Posts: 1333
- Joined: Tue Aug 18, 2009 10:35 pm
- Location: Dallas, Texas
Re: performance troubleshooting/tuning between NL and FC
I like the question, thanks for asking!
So if you are lucky enough to have AO licensed, you can let the system do alllll that work for you... it will move the blocks of data around so that the busy ones are on FC and the not so busy ones stay on NL. This is also called sub-volume-tiering as 1 LUN/Datastare will be a mix-mash of some FC and some NL, depending whats justifies the IO. You can even add some SSD into the mix for better results... in the world of AO, you probably would let 3PAR do the work and turn off your VM features that try to auto-balance and auto storage vmotion resources.
Now for the rest of us that don't have that nifty feature...
SATA/NL is ideal at sequential large block IO, and not ideal at random small block IO, and is cheap.
FC/SCSI is better at random small block IO and is more expensive.
Light workloads *can* work well on NL, especially when you have high spindle counts and lots of cache. Generically speaking though, you will probably want to balance it such that your databases and app data are on FC and your file/print/backups/bin are on NL. That is generic advice, if you have a HEAVY loaded file server, you will want that on FC. If you have a database server that just runs archived historical read only partitions of databases, you may want that on NL.. there are exceptions for every rule.
For data warehousing apps, I have found that building cubes/aggregates is best done on FC (SSD), but once the cube is built for the last and final time, it can be moved to NL. Our typical environment is a cube for current year on FC which gets reprocessed weekly, and previous years that never get edited get pushed down to NL.
Hope this helps!
So if you are lucky enough to have AO licensed, you can let the system do alllll that work for you... it will move the blocks of data around so that the busy ones are on FC and the not so busy ones stay on NL. This is also called sub-volume-tiering as 1 LUN/Datastare will be a mix-mash of some FC and some NL, depending whats justifies the IO. You can even add some SSD into the mix for better results... in the world of AO, you probably would let 3PAR do the work and turn off your VM features that try to auto-balance and auto storage vmotion resources.
Now for the rest of us that don't have that nifty feature...
SATA/NL is ideal at sequential large block IO, and not ideal at random small block IO, and is cheap.
FC/SCSI is better at random small block IO and is more expensive.
Light workloads *can* work well on NL, especially when you have high spindle counts and lots of cache. Generically speaking though, you will probably want to balance it such that your databases and app data are on FC and your file/print/backups/bin are on NL. That is generic advice, if you have a HEAVY loaded file server, you will want that on FC. If you have a database server that just runs archived historical read only partitions of databases, you may want that on NL.. there are exceptions for every rule.
For data warehousing apps, I have found that building cubes/aggregates is best done on FC (SSD), but once the cube is built for the last and final time, it can be moved to NL. Our typical environment is a cube for current year on FC which gets reprocessed weekly, and previous years that never get edited get pushed down to NL.
Hope this helps!
Richard Siemers
The views and opinions expressed are my own and do not necessarily reflect those of my employer.
The views and opinions expressed are my own and do not necessarily reflect those of my employer.
Re: performance troubleshooting/tuning between NL and FC
Thanks for your reply, and unfortunately, no, we don't have AO.
What's interesting is that the DB we are talking about was originally on an F200 striped across 64NL drives and performance seemed fine. Then we moved it to our T800 with 192NL drives, and my admins are saying that performance is less than it was previously. So they moved it to FC.
I know that the T800's NL drives are being hammered by more systems and with more diverse workloads than the F200, but looking at IOPS and port contention, we don't see any issues. I'm curious what are some things that I should look at that might better help us track down where contention is? I'm looking at PD performance and the Service Time, IO Size and such. I'm not really sure what are good ranges, or where one is relevant versus another. Or if PD perf is even all that effective of a place to start.
Is there a good guide to using system reporter...not sure the raw info in the PDF provides a good method of interpreting results.
What's interesting is that the DB we are talking about was originally on an F200 striped across 64NL drives and performance seemed fine. Then we moved it to our T800 with 192NL drives, and my admins are saying that performance is less than it was previously. So they moved it to FC.
I know that the T800's NL drives are being hammered by more systems and with more diverse workloads than the F200, but looking at IOPS and port contention, we don't see any issues. I'm curious what are some things that I should look at that might better help us track down where contention is? I'm looking at PD performance and the Service Time, IO Size and such. I'm not really sure what are good ranges, or where one is relevant versus another. Or if PD perf is even all that effective of a place to start.
Is there a good guide to using system reporter...not sure the raw info in the PDF provides a good method of interpreting results.