HPE Storage Users Group

A Storage Administrator Community




Post new topic Reply to topic  [ 8 posts ] 
Author Message
 Post subject: Upgrades - CPU over 50%
PostPosted: Tue Nov 16, 2021 9:32 am 

Joined: Fri Feb 21, 2020 1:08 pm
Posts: 7
So here's my issue.
I have two Primera's in full FC synchronous replication.
The CPU performance is above 50% (averages close to 60-80% most of the time)
As long as things work (and they do very well) I don't mind.
MAJOR! issue you can't upgrade/update if the CPU is above 50%.

HPe support had me turn off all external services (ie vcenter connection etc) still to high. Their current recommendation is to move stuff off the array to do the damn upgrade. As we were working on a 2nd tier of external storage anyway no Biggy. But this is still messed up.

I'm going to probably end up calling them again to see what I need to move, and or what luns are causing the highCPU. Most (if not all) of the luns are using dedup/compression, and I can see some of them causing higher CPU because their are TB of data that are just images.

Any suggestions?


Top
 Profile  
Reply with quote  
 Post subject: Re: Upgrades - CPU over 50%
PostPosted: Wed Nov 17, 2021 12:58 am 

Joined: Mon Sep 21, 2015 2:11 pm
Posts: 1570
Location: Europe
What is the hardware config?

As these are doing replication I'm guess in a DR case one Primera will take all the load. How would that work with the current CPU utilization?

_________________
The views and opinions expressed are my own and do not necessarily reflect those of my current or previous employers.


Top
 Profile  
Reply with quote  
 Post subject: Re: Upgrades - CPU over 50%
PostPosted: Thu Nov 18, 2021 7:17 pm 

Joined: Fri Feb 21, 2020 1:08 pm
Posts: 7
So this is an A630 all SSD array. ~110TB each
The vmware integration, as well as extra services have been turned off to cut cpu usage at this point.
Currently we use both arrays actively their about 5 blocks from each other with dark FC in between. Synchronous replication so a write to one array is really a write to both. Full DR. it's about 600,000,000+ KB/h avg
Most of the load at the moment is on one array, but even the array that isn't as loaded still gets
I could temporarily stop replication, that would cut I/O's in between the boxes.
The one box is at 15% saturation, and the other around 55%.. most of the time.

As of today Tech Supports answer to the CPU usage, was hey at 7:05 am it was below 50% (I looked it was at 7:05 and 7:10 statistics wise. Then was above that the rest of the day. Their doesn't seem to be a timeframe of 30 minutes when it's below 50% at all. Just lots of blips of one or two stats showing below 50%.
Their recommendation aside from finding a low CPU time of day, was to cut usage to the array during the upgrade. (As a FYI I am currently at 4.2.6 , getting to 4.4.0+ is at least two major upgrades which take 1 1/2 hours each.. (ideally less) ) So I'm supposed to shutdown a chunk of servers for 3-6 hours possibly longer? for a system that's supposed to have almost 100% uptime..
I get maintenance windows.. But wth am I missing.


Top
 Profile  
Reply with quote  
 Post subject: Re: Upgrades - CPU over 50%
PostPosted: Fri Nov 19, 2021 2:51 am 

Joined: Wed May 07, 2014 1:51 am
Posts: 267
You mentioned sync mirroring. Do you have peer persistence setup? You could

    1. Patch/update the array with 15% load
    2. Switchover load to this now fully updated array
    3. Patch/update the other array

_________________
When all else fails, read the instructions.


Top
 Profile  
Reply with quote  
 Post subject: Re: Upgrades - CPU over 50%
PostPosted: Thu Dec 16, 2021 2:14 pm 

Joined: Wed Mar 11, 2015 2:16 pm
Posts: 49
The prechecks should only warn when the CPU % is over 50% but under 75%. You should be able to install the update unless the CPU % is above 75%. Is that not what you're seeing? Sometimes, the GUI can make it look like you can't continue with the install, but you should be able to.


Top
 Profile  
Reply with quote  
 Post subject: Re: Upgrades - CPU over 50%
PostPosted: Thu Feb 24, 2022 8:28 pm 

Joined: Fri Feb 21, 2020 1:08 pm
Posts: 7
wbbouldin wrote:
The prechecks should only warn when the CPU % is over 50% but under 75%. You should be able to install the update unless the CPU % is above 75%. Is that not what you're seeing? Sometimes, the GUI can make it look like you can't continue with the install, but you should be able to.


So I've checked with HPe support and for 3par it was 50-75%.. for A630's which are 2 nodes with 2 cpu's each it's 50%.. If it goes over 50% for the 3-5 minutes it's checking cpu stats it fails the precheck.. After the precheck during the actual upgrade it doesn't care.


Top
 Profile  
Reply with quote  
 Post subject: Re: Upgrades - CPU over 50%
PostPosted: Tue Jun 07, 2022 7:47 am 

Joined: Wed Jun 13, 2018 2:31 am
Posts: 33
think your array is undersized...

are you using deco ? if yes then the compression will take the cpu cycles..

we are on 4N A650 ...and support ask us to move RC to pair 2/3 to reduce pair 0/1 load..
convert volumes with high writes from deco to thin


Top
 Profile  
Reply with quote  
 Post subject: Re: Upgrades - CPU over 50%
PostPosted: Sun Jan 15, 2023 11:16 am 
User avatar

Joined: Tue Oct 07, 2014 8:34 am
Posts: 214
Location: Germany
Are both Nodes (Controller) above 50%?
I have seem some Systems that one Node is about 25% and other steady above 50%.
In this Case I reboot the high CPU Node and after both stay at now 25%.
I assume this happen by a hang Service.


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 8 posts ] 


Who is online

Users browsing this forum: No registered users and 10 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group | DVGFX2 by: Matt