HP Announced Inline De-dupe on their all flash 7450 array

[Cleanur]

it's not the memory as that is handled largely by existing look up tables used for thin provisioning. the reason it's flash only today is about time to market and that that is where the market is today.
As above no one else is doing in line to spinning disk, where as with the efficiencies of dedupe the market addressability for all flash.becomes much much wider.

as for the apples for apples comparison i see this on a daily basis. "Oh vender A is just so much cheaper" but that's typically because they've stacked the deck with 4TB.nearline drives. you have to keep in mind everyone operates in the same market space as such pricing should be very similar and if it isn't you should be suspicious.

[hdtvguy]

I'm curious as to why dedupe is SSD only. They say that the ASIC is doing the heavy lifting to make it happen. So wouldn't the storage medium be irrelevant at that point? Unless possibly they need to do a read from disk in order to make the dedupe work, in which case the spinning rust wouldn't be quick enough for dedupe to work well inline without a major performance hit.

Or maybe it's just a marketing thing to attack the other AFA's which have dedupe, and have been possibly costing HP sales in the AFA market, where in the spinning disk market nobody else has inline dedupe so not having it isn't costing HP any money.

The reason explained to me is the dedupe lookups that have to happen. I assume their reference to the ASIC doing the work is that the AISC will do the IO lookup to SSD validate the hash or something. If they did spinning disk those lookups would burden the IO path and be delayed by the disk response.

[hdtvguy]

The difference was HP and Vendor A proposed 16 SSD's, 52 10K, and 24 7.2K (the performance characteristics were derived from an indepth survey of the environment, hence, the spindle count was important). Vendor B comes in with 12 SSD's, 16 10K, and like 52 7.2K. The usable capacity was there, but the only way they were able to undercut HP was to ignore the spec and propose an array with much lower performance characteristics

Even with 3par I am skeptical when too much falls to NL. We have 384 15K 300GB drives, 128 900GB 10K drives and 128 7K 2TB drives, which works out to roughly 100TB/100TB/230TB and I hate relaying on that much capacity in NL and it has shown to be issues at times. Our new 7400 is configured with 20 SSD 920GB and 320 10K 900GB for around 17TB/260TB and only 3.8TB has migrated to SSD and 68TB is on 10K. This array screams because the snaps are on 10K and there is no 7K bottle necks anywhere. With 1.2TB 10K drives if I were building now it would be SSD, 1.2TB 10K and very little if any NL for workload I know I can pin there.

Where we get killed is in our DR side as data just migrates wherever and when we fire up DR the NL latency kills us! Also even DR testing is done off snaps and those then are NL so our lesson learned SPINDLES, SPINDLES and MORE SPINDLES is the answer, not necessarily faster (SSD), but enough that can handle peak loads.

[Cleanur]

Couldn't agree more, but sometimes Customers just know better The fact that some in the industry are more than willing to promote the nirvana of a SSD & Nearline combination (the aim being to fix problems after the deal is done) does everyone a disservice.

[Richard Siemers]

We need CalvinZ to become a regular contributor/hp-rep here - if you see him Richard - get him to join.

Funny you say that, Calvin has been a member here for some time. He is a fan of the forums and directs people with tech questions to us here. I did meet with Calvin during discover, we trekked long distances on foot through some pretty seedy areas in search of $1 beer and oyster shooters. A great time was had.

So - realtime inline dedupe coming in 3.1.4 to all arrays that will support 3.1.4 (sorry, end of the line for T400/800). Only works on SSD CPGs as far as I could get anyone to commit. Part of the secret sauce was the high speed index/table lookup for dedupe area, there was a fancy word for it but I can't find it right now. Oh, and I do believe they said this will be FREE included in the core OS license... The dedupe is a 16k fixed block, on a per CPG basis.

Six 9s guarantee: Any new system with 4 nodes or more can qualify for a contractual 99.9999% uptime guarantee. I heard this is an Industry first... every vendor claims six nines but none until now has a contract with a clearly defined penalty for delivering less.

Free 1TB StoreVirtual license with every Gen8 Proliant server.

They also unveiled EMC --> 3PAR migration feature which touted a 50% easier migration/less work from VNX --> 3PAR than compared to VNX --> VNX2.

[hdtvguy]

Six 9s guarantee: Any new system with 4 nodes or more can qualify for a contractual 99.9999% uptime guarantee. I heard this is an Industry first... every vendor claims six nines but none until now has a contract with a clearly defined penalty for delivering less.

Marketing BS as well. Would love to know what the t&c's of that look like, probably enough loop holes to drive a truck through.

[MDPlatts]

Thanks richard - was anything mentioned about the flash-cache stuff - how it works and when its available ?

I wonder if once its de-duped in SSD whether it would stay de-duped once AO moves the data down a layer or two.

And if flash-cache and de-dupe might help work together to spread the feature into the other tiers if SSD is handling all the incoming writes with the cache and de-duping them and AO moving them down - though obviously only if the original data is still in the SSD.

[Schmoog]

Thanks richard - was anything mentioned about the flash-cache stuff - how it works and when its available ?

I wonder if once its de-duped in SSD whether it would stay de-duped once AO moves the data down a layer or two.

And if flash-cache and de-dupe might help work together to spread the feature into the other tiers if SSD is handling all the incoming writes with the cache and de-duping them and AO moving them down - though obviously only if the original data is still in the SSD.

I 2nd that about the flash cache.

As far as deduped data staying deduped, my guess would be there is no way to make that happen.

Think about it like this:

Let's say you have a 3 tier 7400 with SSD/FC/NL. You provision a VV out of the SSD CPG which has dedupe turned on.

When one of your hosts issues a write, the data is sent down the wire to the 3par. The 3par will accept your data in cache and acknowledge the write. Before flushing the cache to disk, the 3par performs a dedupe lookup against the "High speed index/table". If the lookup is successful (meaning the 16KB block is duplicate data), then rather that write the data, the array writes a pointer and moves on with life.

Fast forward a few hours when no one cares about the UAT copy of the SQL database you just made anymore (not even enough to delete it). AO will kick in, identify that the copy needs to be moved down to FC. AO will read the 128MB region (getting there through the pointer that dedupe created), write the 128MB region to the FC CPG, redirect the meta, and remove the pointer.

So what just happened is that the AO process rehydrated your data.


Unless there is something else going on in the background (some secret sauce that literally no one has ever heard of, because lets remember dedupe isn't exactly a new feature even if it's new to us), I would say once AO kicks in your data will get re-hydrated (at least the data that got moved to a lower tier).

The thing with it being SSD only is IMO kind of silly. I can understand needing SSD's for the index to prevent the dedupe process from being a performance hog, but they could allow the customer to add a pre-determined number of SSD's (let's say 8, or however many is necessary to give dedupe enough space to work with), dedicate those SSD's into an admin VV to be used only for the dedupe index, and then you can turn dedupe on for the whole array (spinning disk included).

Or am I crazy/missing something/completely stupid?

Regarding the 6 nines guarantee, generally I agree that things like that (including the "Get Thin Guarantee" and the "Double VM Density Guarantee") add up to nothing more that marketing nonsense. However if it's true that "Any 4 node array" qualifies, and that that's all there is to it, the fact that HP is willing to put their money where their mouth is to guarantee better reliability will help them win particularly against small VMAX/DMX/Symmetrix deployments where the customer will typically state that they need a particular array to guarantee availability (fat lot of good that did the state of Virginia a few years back though when a combination of EMC tech support incompetence and on-site staff incompetence brought the whole thing to a screeching halt)

It's a pretty much a potshot against EMC (netapp isn't actually renowned for their reliability), but I'll bet it wins them a few deals, particularly in the large array market where that kind of thing makes a difference, and the tiny array market where some people are fooled into thinking they need that kind of availability

[afidel]

My only comment on the 6 9's guarantee is that if there are enough takers then HP will have a hard financial incentive to do thorough QA on code upgrade, because let's face it, even with an A/P 2 node array you can easily get to 5 9's+ on the hardware for the majority of installations, it's bad code that causes the vast majority of outages. Therefore this can only be a good thing for all 3Par customers.

[Richard Siemers]

They did confirm that de-dupe is on a per CPG basis. How or if de-dupe and AO work together was not mentioned... I assumed when they say de-dupe is only on SSD at this time that implies its exempt from AO.

I didn't hear flash-cache mentioned at all, and completely forgot to ask... although I believe it was already leaked that its suppose to be in 3.1.3 MU1?