HPE Storage Users Group
https://3parug.net/

Unusual performance
https://3parug.net/viewtopic.php?f=18&t=578		 Page 1 of 1
Author:  DaveN [ Wed Feb 05, 2014 8:44 pm ]
Post subject:  Unusual performance
At the moment we are running validation on a brand new 3PAR 7400 with 264 x 450GB x 10,000rpm SAS drives in a full 4 node mesh.

This is designed to be our performance storage that backs our primary Oracle and Exchange systems.

What we have noticed is that sequential read is much slower then sequential write, mainly at lower block sizes. Attached is a graph from the 3PAR management console of the particular VV we are testing on.

This is from a server with 4 paths to the 3PAR. There are 2 ports on the server connected to 4 ports on the 3PAR via a 4GB FC fabric.

The test is running 16k blocks from 8 workers in sequential read and then sequential write.

Is this normal behaviour for a 3PAR system? HP refuse to say whether this is expected or not, but tell us that our SAN is correctly configured and show other graphs with much higher trhoughput; e.g. large blocks, random reads and writes etc.

It isn't a "real world" example of our work load but struck as anomalous and worthy of an explanation before we comiisioned it to production.

Any suggestions or explanations?

Thanks,
Dave

Attachment:
File comment: Graph of performance read vs write at 16k blocks with 8 workers.
3parperf.PNG
3parperf.PNG [ 73.12 KiB | Viewed 15319 times ]
Author:  Richard Siemers [ Wed Feb 05, 2014 10:09 pm ]
Post subject:  Re: Unusual performance
Looks like something went haywire with your read test... its showing a block size of 425k, which is further supported by less than 25 iops.

I would recommend re-validating your read workload parameters are 16k io.
Author:  DaveN [ Wed Feb 05, 2014 11:27 pm ]
Post subject:  Re: Unusual performance
I've double checked the test and they are using 16k blocks.

If I pull up the performance graphs for the data ports on the 3PAR the IO size on the port matches the block size I'm using.

However as you noticed the IO size on the VV is much much larger.

Maybe it's pulling back "step size" sized data off the disk to get the 16k requested? Though our step size is 128k so we'd be looking at a rough multiple of 3.5 step's to process each 16k request?

I can't find a neat multiple of our step size, 5+1 redundancy or something else to explain this.

Thanks,
Dave
Author:  Richard Siemers [ Thu Feb 06, 2014 1:34 pm ]
Post subject:  Re: Unusual performance
I suspect thin provisioning mojo, and perhaps cache are at play here. My guess is you might be "read benchmarking" a thin provisioned disk that is mostly empty? Try blasting your test lun full of random data (not zeros) and then re-running the test again to see if it works differently.

Also, what do the stats look like from the host perspective instead of the IMC realtime?
Author:  Cleanur [ Thu Feb 06, 2014 4:41 pm ]
Post subject:  Re: Unusual performance
You should be looking at VLUNs not VV's. The VV numbers include cache and internal IO so are not representative of front end performance.VV numbers are typically used for performance troubleshooting not benchmarking.
Author:  DaveN [ Thu Feb 06, 2014 6:31 pm ]
Post subject:  Re: Unusual performance
Ahhh many thanks!

Now I can see what's happening and have tracked it back to read/write consolidation occurring in the host.

Just running some more testing to confirm my suspicions.

Thanks,
Dave
Page 1 of 1 All times are UTC - 5 hours
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/