HPE Storage Users Group

A Storage Administrator Community




Post new topic Reply to topic  [ 6 posts ] 
Author Message
 Post subject: Unusual performance
PostPosted: Wed Feb 05, 2014 8:44 pm 

Joined: Wed Feb 05, 2014 8:32 pm
Posts: 3
At the moment we are running validation on a brand new 3PAR 7400 with 264 x 450GB x 10,000rpm SAS drives in a full 4 node mesh.

This is designed to be our performance storage that backs our primary Oracle and Exchange systems.

What we have noticed is that sequential read is much slower then sequential write, mainly at lower block sizes. Attached is a graph from the 3PAR management console of the particular VV we are testing on.

This is from a server with 4 paths to the 3PAR. There are 2 ports on the server connected to 4 ports on the 3PAR via a 4GB FC fabric.

The test is running 16k blocks from 8 workers in sequential read and then sequential write.

Is this normal behaviour for a 3PAR system? HP refuse to say whether this is expected or not, but tell us that our SAN is correctly configured and show other graphs with much higher trhoughput; e.g. large blocks, random reads and writes etc.

It isn't a "real world" example of our work load but struck as anomalous and worthy of an explanation before we comiisioned it to production.

Any suggestions or explanations?

Thanks,
Dave

Attachment:
File comment: Graph of performance read vs write at 16k blocks with 8 workers.
3parperf.PNG
3parperf.PNG [ 73.12 KiB | Viewed 15231 times ]


Top
 Profile  
Reply with quote  
 Post subject: Re: Unusual performance
PostPosted: Wed Feb 05, 2014 10:09 pm 
Site Admin
User avatar

Joined: Tue Aug 18, 2009 10:35 pm
Posts: 1328
Location: Dallas, Texas
Looks like something went haywire with your read test... its showing a block size of 425k, which is further supported by less than 25 iops.

I would recommend re-validating your read workload parameters are 16k io.

_________________
Richard Siemers
The views and opinions expressed are my own and do not necessarily reflect those of my employer.


Top
 Profile  
Reply with quote  
 Post subject: Re: Unusual performance
PostPosted: Wed Feb 05, 2014 11:27 pm 

Joined: Wed Feb 05, 2014 8:32 pm
Posts: 3
I've double checked the test and they are using 16k blocks.

If I pull up the performance graphs for the data ports on the 3PAR the IO size on the port matches the block size I'm using.

However as you noticed the IO size on the VV is much much larger.

Maybe it's pulling back "step size" sized data off the disk to get the 16k requested? Though our step size is 128k so we'd be looking at a rough multiple of 3.5 step's to process each 16k request?

I can't find a neat multiple of our step size, 5+1 redundancy or something else to explain this.

Thanks,
Dave


Top
 Profile  
Reply with quote  
 Post subject: Re: Unusual performance
PostPosted: Thu Feb 06, 2014 1:34 pm 
Site Admin
User avatar

Joined: Tue Aug 18, 2009 10:35 pm
Posts: 1328
Location: Dallas, Texas
I suspect thin provisioning mojo, and perhaps cache are at play here. My guess is you might be "read benchmarking" a thin provisioned disk that is mostly empty? Try blasting your test lun full of random data (not zeros) and then re-running the test again to see if it works differently.

Also, what do the stats look like from the host perspective instead of the IMC realtime?

_________________
Richard Siemers
The views and opinions expressed are my own and do not necessarily reflect those of my employer.


Top
 Profile  
Reply with quote  
 Post subject: Re: Unusual performance
PostPosted: Thu Feb 06, 2014 4:41 pm 

Joined: Wed Aug 07, 2013 3:22 pm
Posts: 254
You should be looking at VLUNs not VV's. The VV numbers include cache and internal IO so are not representative of front end performance.VV numbers are typically used for performance troubleshooting not benchmarking.


Top
 Profile  
Reply with quote  
 Post subject: Re: Unusual performance
PostPosted: Thu Feb 06, 2014 6:31 pm 

Joined: Wed Feb 05, 2014 8:32 pm
Posts: 3
Ahhh many thanks!

Now I can see what's happening and have tracked it back to read/write consolidation occurring in the host.

Just running some more testing to confirm my suspicions.

Thanks,
Dave


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 6 posts ] 


Who is online

Users browsing this forum: Baidu [Spider] and 174 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group | DVGFX2 by: Matt