HP Announced Inline De-dupe on their all flash 7450 array

[Richard Siemers]

At todays launch of HP Discover, David Scott announced All flash storage at the price of spinning disk using $2 per gb as his target. Key technologies behind that claim appear to be inline de-dupe that leverages Gen4 ASICs and 1.9 TB MLC drives.

Unfortunately, I did not get to see the presentation first hand despite being pre-registered and confirmed for the event... it ended up being first come-first serve due to fire code limits on maximum occupancy. So I headed straight for the storage booth for some further clarifications. At this time inline deduce is only available on the 7450 and I was told it is built-in/included in the core OS of the 7450. No word yet as to when we can expect this tech to hit our conventional mixed disk arrays.

[mujzeptu]

Thanks for the update! Any other information you can get is appreciated!

Do you know of the 1.9TB SSDs will be available on the 7400 array or the V400? I am not seeing in the quickspecs for any other array BUT the 7450.... :/

[Davidkn]

I saw a press release about this today, there's also a new flash white paper, will find a link.

The press release is here though, 6 nines of availability too. The inline dedupe is coming in September as an included license of the os suite.

Be interesting to know why this hasn't been announced for any other box. Performance reasons?

http://www.hp.com/hpinfo/newsroom/press ... lFlash.pdf

Here's the optimised for flash white paper too, which includes some more details and explanations.

http://h20195.www2.hp.com/V2/GetPDF.asp ... 264ENW.pdf

No mention of flash cache yet though.....

[Davidkn]

Nice video of the 6 nines program, seems to be across all 4 node systems with a free assessment.

http://m.hp.com/h20621/video-gallery/us ... alk/video/

[Richard Siemers]

Be interesting to know why this hasn't been announced for any other box. Performance reasons?

Possible, but my guess is the guys in marketing drew this line. I wonder how the Storeonce team feels about an inline deduping 7200 I wonder what the difference is between the de-dupe algorithms between the two. Block size? fixed/variable? etc.

[Davidkn]

a colleague of mine seems to thing they are using the extra processor cores of the 7450 to do this, which would kind of make sense, seeing as the ASIC hasn't changed, and would explain why the 7400 and 7200 haven't also been announced.

With the pressure from NetApp and EMC and all the other vendors with dedupe, it was inevitable that HP would eventually have to give in and provide the feature (after sticking to the whole 'not having dedupe on primary data' for so long).

[Davidkn]

However, saying that, the whitepaper explains that its using the ASIC.

Thin de-duplication with HP 3PAR StoreServ ASIC
HP 3PAR StoreServ 7450 Storage employs purpose-built HP 3PAR ASIC at the heart of each controller node that feature
efficient, silicon-based mechanisms to drive inline deduplication. This implementation relies on the 3PAR ASIC to generate
and assign hash signatures to each unique incoming write request. HP 3PAR Express Indexing, a mechanism that
accelerates table lookups, is used for ultra-fast detection of duplicate write requests in order to prevent duplicate data
from being written.
When a new I/O request comes in, HP 3PAR Express Indexing performs instant metadata lookups in order to compare the
signatures of the incoming request to signatures of data already stored in the array. When a match is found, 3PAR
Express Indexing flags the duplicate request and prevents it from being written to the back end. Instead, a pointer is
added to the metadata table to reference the existing data blocks. To prevent any hash collision, HP 3PAR Thin
Deduplication software relies on the controller node ASICs to perform bit-to-bit comparison before any new write update
is marked as a duplicate.
With HP 3PAR Thin Deduplication software, the CPU-intensive jobs of calculating hash signatures for incoming data and
verifying reads are offloaded to the ASICs, freeing up processor cycles to deliver advanced data services and service I/O
requests. This hardware-assisted approach enables inline deduplication that carries multiple benefits, including increased
capacity efficiency, flash performance protection, and flash media lifespan extension.

[Schmoog]

If I had to guess, I would say the changes made to the 7450 to optimize for flash, as well as the cache and control memory advantage that the 7450 would be why this is implemented there.

I would say that there is a slim possibility that hp is just being conservative with a new technology, and that once they have had it running for awhile and are more comfortable with it, they will push it to the other storeservs.

When netapp first released block dedupe they didn't recommend it for production workloads until the technology was more mature.

[Schmoog]

Of course it could also be possible that hp needed to get inline dedupe on the 7450 to reduce their cost enough to compete with the other AFAs on the market, and that it will never come to the spinning disk arrays because hp has absolutely zero interest in lowering the cost of spinning disk beyond what they already have.

I had hp competing against emc for a client, and a similarly spec'd vnx2 (which was harder to come by than you might think. Emc kept trying to skew the spec so their cost was lower), and the 3par was literally half the price. So why should they introduce a technology which lowers that cost even more when they can sit on the cost advantage they already have and keep a very healthy profit margin?

[afidel]

The $2/GB thing is kindof hogwash unless you can actually get 4:1 or better on dedupe. They've announced $14k list for the 1.9TB cMLC (which is a funny name since it's almost 10x the $/GB of actually consumer flash drives). Add in controllers, licensing, support and even with a healthy discount you're looking at $8-10/GB raw. I'd consider it for my DR location where I'm power and space constrained, but it looks like late Q3 at the earliest for GA of dedupe so it's unlikely I'd be able to get my hands of one in time to check out my actual dedupe rates. Add in that Dell is doing $2-3/GB for Compellent all flash without needing any dedupe magic and as much as I like my 3Par I'm thinking they're out for this round.