HPE Storage Users Group
https://3parug.net/

Adaptive Flash Cache problems on 7400 with 3.2.2 (MU6)
https://3parug.net/viewtopic.php?f=18&t=3175
Page 1 of 1

Author:  richard612 [ Mon Apr 29, 2019 11:25 am ]
Post subject:  Adaptive Flash Cache problems on 7400 with 3.2.2 (MU6)

I've posted this over on the HPE Community but one of their moderators moved it to the wrong forum where it's unlikely to get much attention.

We've recently turned on Adaptive Flash Cache for the first time. Now we're seeing two errors; the first is from Infosite: "High slab space usage detected." The second is in Events & Alerts: "Flash Cache I/O times are slow on node: 3"

I've looked at some performance reports in SSMC. Looking at "user time", node 3 has been busier than the others for days/weeks. However, when we enabled Flash Cache a few days ago the "system time" metric for node 3 spiked above the other nodes and addition to user time.

Looking at Physical Drive Performance, I see that the SSDs do have some IOPs spikes, with write latency spiking to 22ms during one stat sample. All 16 SSD drives chart identically.

The report "Adaptive Flash Cache Performance" shows all four nodes charting pretty much identically. Flash Cache used percentage peaked at 400% roughly 20 hours after enablement but it's now slowly wobbling up and down between 300 to 400%. I would expect the cache to remain full once filled. :?:

There are no other workloads on these SSDs (no CPGs).

Any thoughts?

Author:  Patrick [ Fri May 03, 2019 9:38 pm ]
Post subject:  Re: Adaptive Flash Cache problems on 7400 with 3.2.2 (MU6)

When assigning space for AFC, the high slab will show up if you don't leave a certain percentage space free. AFC does have a hit rate command, so you can see some statistics for how well it's picking up the load.

Write latency generally should be very low, there are some cpu background tasks that we've seen spike up the latency every once and awhile but not to the point of any real concern.

AFC only needs a few SSDs on the 7000 series (no need for 16 SSDs to be assigned). Play around with the space allocations a bit till you find what's best for your environment. On one setup with a 10/45/45 mix of SSD/10k/7k AFC started to plateau around 120GB. I saw better overall performance and then setup a small flash tier for some of the heavy hit databases.

Author:  richard612 [ Fri May 10, 2019 9:04 am ]
Post subject:  Re: Adaptive Flash Cache problems on 7400 with 3.2.2 (MU6)

Thanks. My post finally got HP's attention and they said the high swap usage and high slab usage alerts in Infosight were cosmetic (I'm paraphrasing) and not to worry about them.

I did cut the AFC size from 768GB to 128GB. The hit rate is just as high and the alerts went away. I suspect 3PAR's caching implementation is like many others in that it uses RAM to track the blocks in L2 cache and dropping the size freed up a bunch of RAM.

Page 1 of 1 All times are UTC - 5 hours
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/