Mailing List Archive

AFF and 4:1 guaranteed efficiency
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They
say they grantee it. I have specifically asked what stipulation. In a VDI
environment with linked clones, etc. Sales guy tells me none and even
specifically says they can dedup compressed video or audio (mp3) 4:1.

I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?

Any comments welcome.

--Jordan
Re: AFF and 4:1 guaranteed efficiency [ In reply to ]
On Thu, Sep 22, 2016 at 02:32:30PM -0400, jordan slingerland wrote:
> Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9.
> They say they grantee it. I have specifically asked what
> stipulation. In a VDI environment with linked clones, etc. Sales
> guy tells me none and even specifically says they can dedup
> compressed video or audio (mp3) 4:1.
>
> I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or
> what?
>
> Any comments welcome.
>
> --Jordan

We'll be POC'ing one, so will see. Am skeptical, but don't think it's
like Pure Storage where they also factor in snapshots.

Most likely the guarantee is a carrot, and if they fall short, they'll
give you hardware to make up the difference. Subsequent purchases,
however won't give you the same guarantee so you'll need to buy more
(again, this is also what Pure does).

Other thoughts?

Ray
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
Re: AFF and 4:1 guaranteed efficiency [ In reply to ]
my sales guy basically said the guarantee works as follows - they don't hit
the 4:1 target, they buy you whatever amount of shelf\ssd required to
makeup shortfall.

The magic has to do with what they call "compaction" - writes <4KB (down to
512B/ea) are "compacted" into a single 4K block... this apparently adds to
the std dedupe and compression they use.



On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland <
jordan.slingerland@gmail.com> wrote:

> Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They
> say they grantee it. I have specifically asked what stipulation. In a VDI
> environment with linked clones, etc. Sales guy tells me none and even
> specifically says they can dedup compressed video or audio (mp3) 4:1.
>
> I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or
> what?
>
> Any comments welcome.
>
> --Jordan
>
> _______________________________________________
> Toasters mailing list
> Toasters@teaparty.net
> http://www.teaparty.net/mailman/listinfo/toasters
>
>
Re: AFF and 4:1 guaranteed efficiency [ In reply to ]
Well, just one (thought). Life comes w/o guarantee. :-)

Jokes aside, I agree w Ray van D; logically you can get free lunch once or
twice if there's something in it for the buyer (an opportunity of *some*
sort). But never will anyone buy you lunch for a whole year or more --
unless you did something that made them enter voluntary slavery

/M

On 2016-09-22 20:39, Ray Van Dolson wrote:
> We'll be POC'ing one, so will see. Am skeptical, but don't think it's
> like Pure Storage where they also factor in snapshots.
>
> Most likely the guarantee is a carrot, and if they fall short, they'll
> give you hardware to make up the difference. Subsequent purchases,
> however won't give you the same guarantee so you'll need to buy more
> (again, this is also what Pure does).
>
> Other thoughts?
>
> Ray
> _______________________________________________

_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
Re: AFF and 4:1 guaranteed efficiency [ In reply to ]
On Thu, Sep 22, 2016 at 11:39:09AM -0700, Ray Van Dolson wrote:
> On Thu, Sep 22, 2016 at 02:32:30PM -0400, jordan slingerland wrote:
> > Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9.
> > They say they grantee it. I have specifically asked what
> > stipulation. In a VDI environment with linked clones, etc. Sales
> > guy tells me none and even specifically says they can dedup
> > compressed video or audio (mp3) 4:1.
> >
> > I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or
> > what?
> >
> > Any comments welcome.
> >
> > --Jordan
>
> We'll be POC'ing one, so will see. Am skeptical, but don't think it's
> like Pure Storage where they also factor in snapshots.
>
> Most likely the guarantee is a carrot, and if they fall short, they'll
> give you hardware to make up the difference. Subsequent purchases,
> however won't give you the same guarantee so you'll need to buy more
> (again, this is also what Pure does).
>
> Other thoughts?
>
> Ray

Correction: Looking back at notes I have, the guarantee relies on the
use of inline deduplication, inline compression, "compaction"[1], thin
provisioning as well as factoring in "regular" volume snapshots.

NetApp will help you realize the 4:1 savings given the techniques
described previously and then provide additional capacity if needed.

So, take out the snapshots and thin provisioning and will you see
significantly better savings than with other technologies? Not sure --
I'd guess it'll be pretty similar.

Ray

[1] https://urldefense.proofpoint.com/v2/url?u=http-3A__www.theregister.co.uk_2016_06_13_netapps-5F4kb-5Fblock-5Fwrites-5Fcan-5Fhold-5Fmore-5Fdata_&d=DQIBAg&c=n6-cguzQvX_tUIrZOS_4Og&r=LocN40EG5c3kHsbl3Fv0ZPr8zhg62ZqvBx3NsulACiM&m=dr4OWp4UEaiwuyOtPh5NiF0Ldhq9QU2CjkNd8uQ0XDk&s=oTpVQNNuRU2qfEzaxmW5wpM2n2nljWVDDfwAFO2N5RA&e=

_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
Re: AFF and 4:1 guaranteed efficiency [ In reply to ]
On 2016-09-22 20:44, Mike Gossett wrote:
> my sales guy basically said the guarantee works as follows - they don't hit
> the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup
> shortfall.

*duh*

> The magic has to do with what they call "compaction" - writes <4KB (down to
> 512B/ea) are "compacted" into a single 4K block... this apparently adds to
> the std dedupe and compression they use.

Well, magic or not... the answer *must* be: it depends. Note the expressions
below "if", "can be" etc.

- - -
Currently Data ONTAP writes data to storage media in 4KB blocks
* An I/O or file has less than 4KB of data uses an entire 4KB

Adaptive compression can compress an 8KB I/O into a 4KB block on storage
* If the I/O is >50% compressible it is stored in a 4KB block – maximum
storage efficiency ratio is 2:1

Storage space savings can be increased considerably if multiple small I/Os
or files can be stored together in a 4KB block

* Data ONTAP 9 does this with inline data compaction
* Turned on by default on AFF; can be turned on manually for hybrid systems



/M
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
Re: AFF and 4:1 guaranteed efficiency [ In reply to ]
I couldn’t find the document, but I believe NA has updated/upgraded dedupe/compression algorithms for CDOT9. I think it would be “interesting” if you could compress already compressed images/videos. They are typically not candidates for advanced storage features. I am guessing at Insight (next week in US), they will show some of this. I think the guarantee is more being in parity with other storage OEMs.

I think you would have a “discussion” with your Netapp/Sales team before you purchased anything.

http://www.netapp.com/us/forms/sales-inquiry/flash-3-4-5-promotion.aspx


On 9/22/16, 11:39 AM, "toasters-bounces@teaparty.net on behalf of Ray Van Dolson" <toasters-bounces@teaparty.net on behalf of rvandolson@esri.com> wrote:

On Thu, Sep 22, 2016 at 02:32:30PM -0400, jordan slingerland wrote:
> Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9.
> They say they grantee it. I have specifically asked what
> stipulation. In a VDI environment with linked clones, etc. Sales
> guy tells me none and even specifically says they can dedup
> compressed video or audio (mp3) 4:1.
>
> I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or
> what?
>
> Any comments welcome.
>
> --Jordan

We'll be POC'ing one, so will see. Am skeptical, but don't think it's
like Pure Storage where they also factor in snapshots.

Most likely the guarantee is a carrot, and if they fall short, they'll
give you hardware to make up the difference. Subsequent purchases,
however won't give you the same guarantee so you'll need to buy more
(again, this is also what Pure does).

Other thoughts?

Ray
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters



_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
Re: AFF and 4:1 guaranteed efficiency [ In reply to ]
On 2016-09-22 20:54, Klise, Steve wrote:
> I think it would be
> “interesting” if you could compress already compressed images/videos.
> They are typically not candidates for advanced storage features.

You cannot really, since the Shannon Entropy of such files is pretty much
maxed out already.

<https://en.wikipedia.org/wiki/Entropy_(information_theory)>

But let's say you have a large no of smaller video files, sitting in an
ONTAP (WAFL) based system. This 4K block factor, the new "Compaction" will
give a positive effect.

Again: it depends

/M
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
Re: AFF and 4:1 guaranteed efficiency [ In reply to ]
Those sales folks may be misunderstanding the details.
Ask to see the details of the guarantee, and read for yourself.

My experience with CDOT 8 on a AFF8020 was that video didn't compress/dedupe
hardly at all (12% savings from dedupe, 0% from compression). Student home directories
are getting about 48% savings (close to 2:1).

I imagine certain data stores will have much better savings (like
lots of copies of VMs with identical OS images on them).

NetApp has a tool you can use to estimate savings of your own data,
though it caps the size of the analysis at 2TB (regardless of the size of the
volume being analyzed). They call it SSET (Space Savings Estimation Tool).
I'm not sure whether that's specific to CDOT 8 or not.

Re:
> From: jordan slingerland <jordan.slingerland@gmail.com>
> Date: Thu, 22 Sep 2016 14:32:30 -0400
> Subject: AFF and 4:1 guaranteed efficiency
> To: toasters@teaparty.net
>
> Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They
> say they grantee it. I have specifically asked what stipulation. In a VDI
> environment with linked clones, etc. Sales guy tells me none and even
> specifically says they can dedup compressed video or audio (mp3) 4:1.
>
> I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?
>
> Any comments welcome.
>
> --Jordan

> _______________________________________________
> Toasters mailing list
> Toasters@teaparty.net
> http://www.teaparty.net/mailman/listinfo/toasters


--
Brian Parent
Information Technology Services Department
IT Infrastructure Operations Group
Workplace, Internal, Research, and Educational Platforms (WIRE) team
UC San Diego
(858) 534-6090
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
Re: AFF and 4:1 guaranteed efficiency [ In reply to ]
>>hose sales folks may be misunderstanding the details.
>>Ask to see the details of the guarantee, and read for yourself.

I agree. Thanks for all the responses, much more inline with what I would
think. My data is not video or audio, but I did bring that up to the sales
guys and very specifically they said they could get 4:1 on it. In fact the
only example dataset they had any hesitation on promising verbally was the
output of /dev/urandom. I call BS. Don't get me wrong, ontap9 on AFF
sounds like it contains a lot of great incremental improvement and I am
excited to try one out. I just prefer realistic promises.

--Jordan

On Thu, Sep 22, 2016 at 3:07 PM, Brian Parent <bparent@ucsd.edu> wrote:

> Those sales folks may be misunderstanding the details.
> Ask to see the details of the guarantee, and read for yourself.
>
> My experience with CDOT 8 on a AFF8020 was that video didn't
> compress/dedupe
> hardly at all (12% savings from dedupe, 0% from compression). Student
> home directories
> are getting about 48% savings (close to 2:1).
>
> I imagine certain data stores will have much better savings (like
> lots of copies of VMs with identical OS images on them).
>
> NetApp has a tool you can use to estimate savings of your own data,
> though it caps the size of the analysis at 2TB (regardless of the size of
> the
> volume being analyzed). They call it SSET (Space Savings Estimation Tool).
> I'm not sure whether that's specific to CDOT 8 or not.
>
> Re:
> > From: jordan slingerland <jordan.slingerland@gmail.com>
> > Date: Thu, 22 Sep 2016 14:32:30 -0400
> > Subject: AFF and 4:1 guaranteed efficiency
> > To: toasters@teaparty.net
> >
> > Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They
> > say they grantee it. I have specifically asked what stipulation. In a
> VDI
> > environment with linked clones, etc. Sales guy tells me none and even
> > specifically says they can dedup compressed video or audio (mp3) 4:1.
> >
> > I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or
> what?
> >
> > Any comments welcome.
> >
> > --Jordan
>
> > _______________________________________________
> > Toasters mailing list
> > Toasters@teaparty.net
> > http://www.teaparty.net/mailman/listinfo/toasters
>
>
> --
> Brian Parent
> Information Technology Services Department
> IT Infrastructure Operations Group
> Workplace, Internal, Research, and Educational Platforms (WIRE) team
> UC San Diego
> (858) 534-6090
>
Re: AFF and 4:1 guaranteed efficiency [ In reply to ]
On Thu, Sep 22, 2016 at 03:21:02PM -0400, jordan slingerland wrote:
> >>hose sales folks may be misunderstanding the details.
> >>Ask to see the details of the guarantee, and read for yourself.
>
> I agree. Thanks for all the responses, much more inline with what I would
> think. My data is not video or audio, but I did bring that up to the sales
> guys and very specifically they said they could get 4:1 on it. In fact the
> only example dataset they had any hesitation on promising verbally was the
> output of /dev/urandom. I call BS. Don't get me wrong, ontap9 on AFF sounds
> like it contains a lot of great incremental improvement and I am excited to
> try one out. I just prefer realistic promises.
>
> --Jordan

As other's have mentioned, it's likely more about matching up with the
marketing of competitors. Just about everyone has a data reduction
guarantee now with the same asterisks and strings attached.

Ray
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
Re: AFF and 4:1 guaranteed efficiency [ In reply to ]
As it was mentioned before, you can use SSET to measure efficiency of
NetApp technologies with your dataset. Last version of SSET support inline
data compaction. You can download SSET from Utility Toolchest
http://mysupport.netapp.com/tools/index.html

And here is some information about the guarantee:

"The total effective storage capacity ratio is calculated based on
compression, deduplication, compaction, snapshots and clones on the dataset.

Although all volumes (new and migrated ones) need to be thinly provisioned,
thin provisioning is not included in measuring the total effective storage
capacity ratio."

"The guarantee is for all workloads running on All Flash FAS systems. The
AFF Systems can be added to the new cluster provided they adhere to the
NetApp Storage Efficiency technologies. However, already compressed data
and encrypted data is excluded from the guarantee. "


Just ask sales to show you program Terms and Conditions.

On Thu, Sep 22, 2016 at 10:27 PM, Ray Van Dolson <rvandolson@esri.com>
wrote:

> On Thu, Sep 22, 2016 at 03:21:02PM -0400, jordan slingerland wrote:
> > >>hose sales folks may be misunderstanding the details.
> > >>Ask to see the details of the guarantee, and read for yourself.
> >
> > I agree. Thanks for all the responses, much more inline with what I
> would
> > think. My data is not video or audio, but I did bring that up to the
> sales
> > guys and very specifically they said they could get 4:1 on it. In fact
> the
> > only example dataset they had any hesitation on promising verbally was
> the
> > output of /dev/urandom. I call BS. Don't get me wrong, ontap9 on AFF
> sounds
> > like it contains a lot of great incremental improvement and I am
> excited to
> > try one out. I just prefer realistic promises.
> >
> > --Jordan
>
> As other's have mentioned, it's likely more about matching up with the
> marketing of competitors. Just about everyone has a data reduction
> guarantee now with the same asterisks and strings attached.
>
> Ray
> _______________________________________________
> Toasters mailing list
> Toasters@teaparty.net
> http://www.teaparty.net/mailman/listinfo/toasters
>
RE: AFF and 4:1 guaranteed efficiency [ In reply to ]
I honestly don't know the terms of the guarantee at this time, but here's a few observations on this topic of efficiencies and guarantees:


1) Always read the remedies for the guarantees. They seem mostly the same to me across the vendors.

2) There's a reason for that - the efficiency rates are more or less the same for any of the various products on the market.

I've noted a few competitors who tend to cite 4:1, 5:1, or 6:1 without any qualification. On occasion, I've heard of one of them having to send lots of extra shelves of disks to make up for the shortfall, but most of the time the answer is "you had a nonstandard configuration" which is a different way of saying "too bad".

Most customers I know do see about 4:1 efficiencies when they properly implement an ONTAP 9 system, but obviously there will be variations. VDI is usually pretty easy because the VMDK's deduplicate extremely well. I've seen databases that were mostly empty and compressed and compacted 80:1. If someone is in the habit of storing loads of zeros in their databases, we'd blow that 4:1 out of the water, but that's a fringe case. It's due to compaction. All those 8K blocks are composed of a small header and trailer sandwiching some zeros. The zeros compress mostly out of existence and compaction will then store the residual header/trailer data into a tiny space. Fringe case, but not out of the question. On the flipside, I ran into a customer who was getting zero compression because their DBA's had enabled encryption across the board without telling the storage team. They were getting no savings at all. Still, 4:1 is about the typical level in my experience with ONTAP.

There's also more than just compression and efficiency. I know one customer who took a 60TB database and cloned it 40 times. That's 40:1 right there without compression or deduplication. You could do the same thing on some competing storage arrays and the clone savings would be done via deduplication, whereas we do it via direct cloning of a snapshot. It's quicker, but it's not classified as "efficiency" because it's not done via deduplication or cloning. It sometimes isn't getting the deserved credit as an efficiency option.

It's also worth mentioning the big picture. For one, the cost of SSD's keeps dropping, which erodes the value of efficiency. The cost savings are less and less all the time. Maybe some product out there does have 4:1 efficiency on a certain data set while ONTAP only had 3:1, but how much money does that really save, and what are the consequences? There are more features and needs beyond space savings. For example, some products use larger compression block sizes and get better efficiency. You could even force this with ONTAP via secondary compression which uses a larger block size and therefore delivers better efficiency levels. Unfortunately, with data sets like VDI and databases that involve small block overwrites you will see a latency hit. So, better efficiency with worse performance.

From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of jordan slingerland
Sent: Thursday, September 22, 2016 1:33 PM
To: toasters@teaparty.net
Subject: AFF and 4:1 guaranteed efficiency

Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.
I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?
Any comments welcome.
--Jordan
Re: AFF and 4:1 guaranteed efficiency [ In reply to ]
Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else.  (SSET diagnosis)

However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.

It must transfer, as far as we're told today, via a host/file based migration.   Be thinking bout this when you consider an AFF migration.  It must be done outside on ONTAP.

We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.
 _________________________________Jeff MohlerTech Yahoo, Storage Architect, Principal(831)454-6712
YPAC Gold Member
Twitter: @PrincipalYahoo
CorpIM:  Hipchat & Iris



On Friday, September 23, 2016 1:53 AM, Mike Gossett <cmgossett@gmail.com> wrote:


my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.  
The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.


On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland <jordan.slingerland@gmail.com> wrote:

Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it.  I have specifically asked what stipulation.  In a VDI environment with linked clones, etc.  Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.

I have A LOT of trouble believing that.  So, 4:1 dedup guaranteed or what? 

Any comments welcome.

--Jordan

______________________________ _________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/ mailman/listinfo/toasters




_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
Re: AFF and 4:1 guaranteed efficiency [ In reply to ]
On 2016-09-23 09:09, Jeffrey Mohler wrote:
> Compaction in our testing can be good, really good...an additional 20% in
> many of our test data sets on top of everything else. (SSET diagnosis)

I'm not too surprised (w.r.t. Yahoo). I expect much the same for us here

> However, there are limitations on how you get data into AFF to get
> compaction...IE, you cant snapmirror it.
>
> It must transfer, as far as we're told today, via a host/file based
> migration. Be thinking bout this when you consider an AFF migration. It must
> be done outside on ONTAP.
>
> We have pushed to get this [having to file migrate via protocol] fixed
> via a scanner/etc that reads and relays out the file based structure.

We = Yahoo? You've requested NetApp to implement a bg WAFL scanner which
will runt through the whole FlexVol (vol by vol) and compact it?
While this will prob be fine (-ish) for AFF I don't think that scanner will
be pleasant, even viable, to run on HyA based systems (spinning disk). If
they do this my hunch is it will be limited to run in AFF only...

/M

_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
Re: AFF and 4:1 guaranteed efficiency [ In reply to ]
A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move.

Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import.

Sent from my mobile phone.

On 23 Sep 2016, at 02:10, Jeffrey Mohler <jmohler@yahoo-inc.com<mailto:jmohler@yahoo-inc.com>> wrote:

Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else. (SSET diagnosis)

However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.

It must transfer, as far as we're told today, via a host/file based migration. Be thinking bout this when you consider an AFF migration. It must be done outside on ONTAP.

We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.


_________________________________
Jeff Mohler<mailto:jmohler@yahoo-inc.com>
Tech Yahoo, Storage Architect, Principal
(831)454-6712
YPAC Gold Member
Twitter: @PrincipalYahoo
CorpIM: Hipchat & Iris



On Friday, September 23, 2016 1:53 AM, Mike Gossett <cmgossett@gmail.com<mailto:cmgossett@gmail.com>> wrote:


my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.

The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.



On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland <jordan.slingerland@gmail.com<mailto:jordan.slingerland@gmail.com>> wrote:
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.

I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?

Any comments welcome.

--Jordan

______________________________ _________________
Toasters mailing list
Toasters@teaparty.net<mailto:Toasters@teaparty.net>
http://www.teaparty.net/ mailman/listinfo/toasters<http://www.teaparty.net/mailman/listinfo/toasters>



_______________________________________________
Toasters mailing list
Toasters@teaparty.net<mailto:Toasters@teaparty.net>
http://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
Toasters@teaparty.net<mailto:Toasters@teaparty.net>
http://www.teaparty.net/mailman/listinfo/toasters
Re: AFF and 4:1 guaranteed efficiency [ In reply to ]
Strange how compaction appears to work with vol move but not with snap mirror.

.

On Sep 23, 2016, at 4:39 AM, Steiner, Jeffrey <Jeffrey.Steiner@netapp.com<mailto:Jeffrey.Steiner@netapp.com>> wrote:

A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move.

Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import.

Sent from my mobile phone.

On 23 Sep 2016, at 02:10, Jeffrey Mohler <jmohler@yahoo-inc.com<mailto:jmohler@yahoo-inc.com>> wrote:

Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else. (SSET diagnosis)

However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.

It must transfer, as far as we're told today, via a host/file based migration. Be thinking bout this when you consider an AFF migration. It must be done outside on ONTAP.

We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.


_________________________________
Jeff Mohler<mailto:jmohler@yahoo-inc.com>
Tech Yahoo, Storage Architect, Principal
(831)454-6712
YPAC Gold Member
Twitter: @PrincipalYahoo
CorpIM: Hipchat & Iris



On Friday, September 23, 2016 1:53 AM, Mike Gossett <cmgossett@gmail.com<mailto:cmgossett@gmail.com>> wrote:


my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.

The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.



On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland <jordan.slingerland@gmail.com<mailto:jordan.slingerland@gmail.com>> wrote:
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.

I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?

Any comments welcome.

--Jordan

______________________________ _________________
Toasters mailing list
Toasters@teaparty.net<mailto:Toasters@teaparty.net>
http://www.teaparty.net/ mailman/listinfo/toasters<http://www.teaparty.net/mailman/listinfo/toasters>



_______________________________________________
Toasters mailing list
Toasters@teaparty.net<mailto:Toasters@teaparty.net>
http://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
Toasters@teaparty.net<mailto:Toasters@teaparty.net>
http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________
Toasters mailing list
Toasters@teaparty.net<mailto:Toasters@teaparty.net>
http://www.teaparty.net/mailman/listinfo/toasters
Re: AFF and 4:1 guaranteed efficiency [ In reply to ]
My understanding is that both are block based operations so it makes sense
to me that the blocks would be put down on disk unchanged. Perhaps it has
something to do with the fact that you may want to reverse the snapmirror
back to the system that presumably does not have compaction enabled or
support the feature, so the blocks are left uncompacted? If the vol move
is within the same controller AFF the lack of backward compatibility is not
a concern. Just a though, I don't know.

On Fri, Sep 23, 2016 at 10:57 AM, Francis Kim <fkim@berkcom.com> wrote:

> Strange how compaction appears to work with vol move but not with snap
> mirror.
>
> .
>
> On Sep 23, 2016, at 4:39 AM, Steiner, Jeffrey <Jeffrey.Steiner@netapp.com>
> wrote:
>
> A vol-move operation will cause compaction to happen. I know it's not
> ideal but at least its internal. Obviously a scanner is preferred but
> functionally it would be a lot like a vol move.
>
> Likewise if you use FLI to import a LUN that will trigger all the
> efficiency features during the import.
>
> Sent from my mobile phone.
>
> On 23 Sep 2016, at 02:10, Jeffrey Mohler <jmohler@yahoo-inc.com> wrote:
>
> Compaction in our testing can be good, really good...an additional 20% in
> many of our test data sets on top of everything else. (SSET diagnosis)
>
> However, there are limitations on how you get data into AFF to get
> compaction...IE, you cant snapmirror it.
>
> It must transfer, as far as we're told today, via a host/file based
> migration. Be thinking bout this when you consider an AFF migration. It
> must be done outside on ONTAP.
>
> We have pushed to get this fixed via a scanner/etc that reads and relays
> out the file based structure.
>
>
> _________________________________
> Jeff Mohler <jmohler@yahoo-inc.com>
> Tech Yahoo, Storage Architect, Principal
> (831)454-6712
> YPAC Gold Member
> Twitter: @PrincipalYahoo
> CorpIM: Hipchat & Iris
>
>
>
> On Friday, September 23, 2016 1:53 AM, Mike Gossett <cmgossett@gmail.com>
> wrote:
>
>
> my sales guy basically said the guarantee works as follows - they don't
> hit the 4:1 target, they buy you whatever amount of shelf\ssd required to
> makeup shortfall.
>
> The magic has to do with what they call "compaction" - writes <4KB (down
> to 512B/ea) are "compacted" into a single 4K block... this apparently adds
> to the std dedupe and compression they use.
>
>
>
> On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland <
> jordan.slingerland@gmail.com> wrote:
>
> Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They
> say they grantee it. I have specifically asked what stipulation. In a VDI
> environment with linked clones, etc. Sales guy tells me none and even
> specifically says they can dedup compressed video or audio (mp3) 4:1.
>
> I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or
> what?
>
> Any comments welcome.
>
> --Jordan
>
> ______________________________ _________________
> Toasters mailing list
> Toasters@teaparty.net
> http://www.teaparty.net/ mailman/listinfo/toasters
> <http://www.teaparty.net/mailman/listinfo/toasters>
>
>
>
> _______________________________________________
> Toasters mailing list
> Toasters@teaparty.net
> http://www.teaparty.net/mailman/listinfo/toasters
>
>
> _______________________________________________
> Toasters mailing list
> Toasters@teaparty.net
> http://www.teaparty.net/mailman/listinfo/toasters
>
> _______________________________________________
> Toasters mailing list
> Toasters@teaparty.net
> http://www.teaparty.net/mailman/listinfo/toasters
>
>
> _______________________________________________
> Toasters mailing list
> Toasters@teaparty.net
> http://www.teaparty.net/mailman/listinfo/toasters
>
>
RE: AFF and 4:1 guaranteed efficiency [ In reply to ]
I would guess that snapmirror is currently looking at the physical blocks on disk, whereas a vol-move is a step up the chain and it can look at block that are actually a block within a block.

I'll check with engineering and report back...

From: jordan slingerland [mailto:jordan.slingerland@gmail.com]
Sent: Friday, September 23, 2016 10:26 AM
To: NGC-fkim-berkcom.com <fkim@berkcom.com>
Cc: Steiner, Jeffrey <Jeffrey.Steiner@netapp.com>; toasters@teaparty.net
Subject: Re: AFF and 4:1 guaranteed efficiency

My understanding is that both are block based operations so it makes sense to me that the blocks would be put down on disk unchanged. Perhaps it has something to do with the fact that you may want to reverse the snapmirror back to the system that presumably does not have compaction enabled or support the feature, so the blocks are left uncompacted? If the vol move is within the same controller AFF the lack of backward compatibility is not a concern. Just a though, I don't know.

On Fri, Sep 23, 2016 at 10:57 AM, Francis Kim <fkim@berkcom.com<mailto:fkim@berkcom.com>> wrote:
Strange how compaction appears to work with vol move but not with snap mirror.

.

On Sep 23, 2016, at 4:39 AM, Steiner, Jeffrey <Jeffrey.Steiner@netapp.com<mailto:Jeffrey.Steiner@netapp.com>> wrote:
A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move.

Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import.

Sent from my mobile phone.

On 23 Sep 2016, at 02:10, Jeffrey Mohler <jmohler@yahoo-inc.com<mailto:jmohler@yahoo-inc.com>> wrote:
Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else. (SSET diagnosis)

However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.
It must transfer, as far as we're told today, via a host/file based migration. Be thinking bout this when you consider an AFF migration. It must be done outside on ONTAP.

We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.


_________________________________
Jeff Mohler<mailto:jmohler@yahoo-inc.com>
Tech Yahoo, Storage Architect, Principal
(831)454-6712
YPAC Gold Member
Twitter: @PrincipalYahoo
CorpIM: Hipchat & Iris


On Friday, September 23, 2016 1:53 AM, Mike Gossett <cmgossett@gmail.com<mailto:cmgossett@gmail.com>> wrote:

my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.

The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.



On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland <jordan.slingerland@gmail.com<mailto:jordan.slingerland@gmail.com>> wrote:
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.
I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?
Any comments welcome.
--Jordan

______________________________ _________________
Toasters mailing list
Toasters@teaparty.net<mailto:Toasters@teaparty.net>
http://www.teaparty.net/ mailman/listinfo/toasters<http://www.teaparty.net/mailman/listinfo/toasters>


_______________________________________________
Toasters mailing list
Toasters@teaparty.net<mailto:Toasters@teaparty.net>
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters@teaparty.net<mailto:Toasters@teaparty.net>
http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________
Toasters mailing list
Toasters@teaparty.net<mailto:Toasters@teaparty.net>
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters@teaparty.net<mailto:Toasters@teaparty.net>
http://www.teaparty.net/mailman/listinfo/toasters
Re: AFF and 4:1 guaranteed efficiency [ In reply to ]
I vaguely remember running a test on an ONTAP 9 simulator.

What I found was that if the destination aggr had compaction enabled, then
snapmirror into that aggr would also have compacted data.
This DOES NOT work with XDP (snapvault & version-flexible snapmirror)



--tmac

*Tim McCarthy, **Principal Consultant*

*Proud Member of the #NetAppATeam <https://twitter.com/NetAppATeam>*

*I Blog at TMACsRack <https://tmacsrack.wordpress.com/>*


[image: NetApp - In partnership with Alpine Testing Solutions][image:
NetApp Certified Data Administrator, ONTAP]
<https://www.certmetrics.com/netapp/public/badge.aspx?t=c&d=2012-11-05&i=35&ci=NETAPP00041276>[image:
NetApp Certified Implementation Engineer - SAN Specialist, ONTAP]
<https://www.certmetrics.com/netapp/public/badge.aspx?t=c&d=2012-11-08&i=36&ci=NETAPP00041276>[image:
NetApp Certified Storage Installation Engineer, ONTAP]
<https://www.certmetrics.com/netapp/public/badge.aspx?t=c&d=2015-10-13&i=38&ci=NETAPP00041276>[image:
NetApp Certified Implementation Engineer - Data Protection Specialist]
<https://www.certmetrics.com/netapp/public/badge.aspx?t=c&d=2015-10-15&i=11&ci=NETAPP00041276>


NetApp Candidate ID: NETAPP00041276
FlexPod Design: Oct 2015 - Jan 2018, S0N62WE1BMVEYF3M
FlexPod Implementation: Oct 2015 - Jan 2018, JH3QJT4KLEQ41HPH

RHCE6 110-107-141
<https://www.redhat.com/wapps/training/certification/verify.html?certNumber=110-107-141&isSearch=False&verify=Verify>


On Fri, Sep 23, 2016 at 11:25 AM, jordan slingerland <
jordan.slingerland@gmail.com> wrote:

> My understanding is that both are block based operations so it makes sense
> to me that the blocks would be put down on disk unchanged. Perhaps it has
> something to do with the fact that you may want to reverse the snapmirror
> back to the system that presumably does not have compaction enabled or
> support the feature, so the blocks are left uncompacted? If the vol move
> is within the same controller AFF the lack of backward compatibility is not
> a concern. Just a though, I don't know.
>
> On Fri, Sep 23, 2016 at 10:57 AM, Francis Kim <fkim@berkcom.com> wrote:
>
>> Strange how compaction appears to work with vol move but not with snap
>> mirror.
>>
>> .
>>
>> On Sep 23, 2016, at 4:39 AM, Steiner, Jeffrey <Jeffrey.Steiner@netapp.com>
>> wrote:
>>
>> A vol-move operation will cause compaction to happen. I know it's not
>> ideal but at least its internal. Obviously a scanner is preferred but
>> functionally it would be a lot like a vol move.
>>
>> Likewise if you use FLI to import a LUN that will trigger all the
>> efficiency features during the import.
>>
>> Sent from my mobile phone.
>>
>> On 23 Sep 2016, at 02:10, Jeffrey Mohler <jmohler@yahoo-inc.com> wrote:
>>
>> Compaction in our testing can be good, really good...an additional 20% in
>> many of our test data sets on top of everything else. (SSET diagnosis)
>>
>> However, there are limitations on how you get data into AFF to get
>> compaction...IE, you cant snapmirror it.
>>
>> It must transfer, as far as we're told today, via a host/file based
>> migration. Be thinking bout this when you consider an AFF migration. It
>> must be done outside on ONTAP.
>>
>> We have pushed to get this fixed via a scanner/etc that reads and relays
>> out the file based structure.
>>
>>
>> _________________________________
>> Jeff Mohler <jmohler@yahoo-inc.com>
>> Tech Yahoo, Storage Architect, Principal
>> (831)454-6712
>> YPAC Gold Member
>> Twitter: @PrincipalYahoo
>> CorpIM: Hipchat & Iris
>>
>>
>>
>> On Friday, September 23, 2016 1:53 AM, Mike Gossett <cmgossett@gmail.com>
>> wrote:
>>
>>
>> my sales guy basically said the guarantee works as follows - they don't
>> hit the 4:1 target, they buy you whatever amount of shelf\ssd required to
>> makeup shortfall.
>>
>> The magic has to do with what they call "compaction" - writes <4KB (down
>> to 512B/ea) are "compacted" into a single 4K block... this apparently adds
>> to the std dedupe and compression they use.
>>
>>
>>
>> On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland <
>> jordan.slingerland@gmail.com> wrote:
>>
>> Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They
>> say they grantee it. I have specifically asked what stipulation. In a VDI
>> environment with linked clones, etc. Sales guy tells me none and even
>> specifically says they can dedup compressed video or audio (mp3) 4:1.
>>
>> I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or
>> what?
>>
>> Any comments welcome.
>>
>> --Jordan
>>
>> ______________________________ _________________
>> Toasters mailing list
>> Toasters@teaparty.net
>> http://www.teaparty.net/ mailman/listinfo/toasters
>> <http://www.teaparty.net/mailman/listinfo/toasters>
>>
>>
>>
>> _______________________________________________
>> Toasters mailing list
>> Toasters@teaparty.net
>> http://www.teaparty.net/mailman/listinfo/toasters
>>
>>
>> _______________________________________________
>> Toasters mailing list
>> Toasters@teaparty.net
>> http://www.teaparty.net/mailman/listinfo/toasters
>>
>> _______________________________________________
>> Toasters mailing list
>> Toasters@teaparty.net
>> http://www.teaparty.net/mailman/listinfo/toasters
>>
>>
>> _______________________________________________
>> Toasters mailing list
>> Toasters@teaparty.net
>> http://www.teaparty.net/mailman/listinfo/toasters
>>
>>
>
> _______________________________________________
> Toasters mailing list
> Toasters@teaparty.net
> http://www.teaparty.net/mailman/listinfo/toasters
>
>
Re: AFF and 4:1 guaranteed efficiency [ In reply to ]
blockquote, div.yahoo_quoted { margin-left: 0 !important; border-left:1px #715FFA solid !important; padding-left:1ex !important; background-color:white !important; } Yes I did.  If I want to and it reduces my costs by 20%,   I want that OPTION.  
If you could run that across say 600PB of data, as one worlkload example for said 20%, you wouldn't?
Even on dense sata only.  I would want that option. 
Gaining even 10% for data at moderate rest for with nearly zero performance guarantees against it would be worth the week, two, or so of scanning.  
It's Netapp's job to make it viable and compete.  Mine (ours) to guide them (and others)on what we all want.  

















Sent from Yahoo Mail for iPhone


On Friday, September 23, 2016, 3:45 PM, Michael Bergman <michael.bergman@ericsson.com> wrote:

On 2016-09-23 09:09, Jeffrey Mohler wrote:
> Compaction in our testing can be good, really good...an additional 20% in
> many of our test data sets on top of everything else. (SSET diagnosis)

I'm not too surprised (w.r.t. Yahoo). I expect much the same for us here

> However, there are limitations on how you get data into AFF to get
> compaction...IE, you cant snapmirror it.
>
> It must transfer, as far as we're told today, via a host/file based
> migration. Be thinking bout this when you consider an AFF migration. It must
> be done outside on ONTAP.
>
> We have pushed to get this [having to file migrate via protocol] fixed
> via  a scanner/etc that reads and relays out  the file based structure.

We = Yahoo? You've requested NetApp to implement a bg WAFL scanner which
will runt through the whole FlexVol (vol by vol) and compact it?
While this will prob be fine (-ish) for AFF I don't think that scanner will
be pleasant, even viable, to run on HyA based systems (spinning disk). If
they do this my hunch is it will be limited to run in AFF only...

/M

_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
Re: AFF and 4:1 guaranteed efficiency [ In reply to ]
blockquote, div.yahoo_quoted { margin-left: 0 !important; border-left:1px #715FFA solid !important; padding-left:1ex !important; background-color:white !important; } Good to know.  Have not received that feedback yet from the team.  
But I've been in Thailand for a week.  


Sent from Yahoo Mail for iPhone


On Friday, September 23, 2016, 6:37 PM, Steiner, Jeffrey <Jeffrey.Steiner@netapp.com> wrote:

A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move. 
Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import. 

Sent from my mobile phone. 
On 23 Sep 2016, at 02:10, Jeffrey Mohler <jmohler@yahoo-inc.com> wrote:


Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else.  (SSET diagnosis)

However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.

It must transfer, as far as we're told today, via a host/file based migration.   Be thinking bout this when you consider an AFF migration.  It must be done outside on ONTAP.

We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.
 _________________________________Jeff MohlerTech Yahoo, Storage Architect, Principal(831)454-6712
YPAC Gold Member
Twitter: @PrincipalYahoo
CorpIM:  Hipchat & Iris



On Friday, September 23, 2016 1:53 AM, Mike Gossett <cmgossett@gmail.com> wrote:


my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.  
The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.


On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland<jordan.slingerland@gmail.com> wrote:

Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it.  I have specifically asked what stipulation.  In a VDI environment with linked clones, etc.  Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.

I have A LOT of trouble believing that.  So, 4:1 dedup guaranteed or what? 

Any comments welcome.

--Jordan

______________________________ _________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/ mailman/listinfo/toasters




_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters




_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
Re: AFF and 4:1 guaranteed efficiency [ In reply to ]
blockquote, div.yahoo_quoted { margin-left: 0 !important; border-left:1px #715FFA solid !important; padding-left:1ex !important; background-color:white !important; } Compaction is block content based.  Not just the block.  
Snap mirror doesn't understand block contents.  


Sent from Yahoo Mail for iPhone


On Friday, September 23, 2016, 10:25 PM, jordan slingerland <jordan.slingerland@gmail.com> wrote:

My understanding is that both are block based operations so it makes sense to me that the blocks would be put down on disk unchanged.  Perhaps it has something to do with the fact that you may want to reverse the snapmirror back to the system that presumably does not have compaction enabled or support the feature, so the blocks are left uncompacted?  If the vol move is within the same controller AFF the lack of backward compatibility is not a concern.  Just a though, I don't know. 

On Fri, Sep 23, 2016 at 10:57 AM, Francis Kim <fkim@berkcom.com> wrote:

Strange how compaction appears to work with vol move but not with snap mirror. 
.
On Sep 23, 2016, at 4:39 AM, Steiner, Jeffrey <Jeffrey.Steiner@netapp.com> wrote:


A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move. 
Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import. 

Sent from my mobile phone. 
On 23 Sep 2016, at 02:10, Jeffrey Mohler <jmohler@yahoo-inc.com> wrote:


Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else.  (SSET diagnosis)

However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.

It must transfer, as far as we're told today, via a host/file based migration.   Be thinking bout this when you consider an AFF migration.  It must be done outside on ONTAP.

We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.
 ______________________________ ___Jeff MohlerTech Yahoo, Storage Architect, Principal(831)454-6712
YPAC Gold Member
Twitter: @PrincipalYahoo
CorpIM:  Hipchat & Iris



On Friday, September 23, 2016 1:53 AM, Mike Gossett <cmgossett@gmail.com> wrote:


my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.  
The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.


On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland<jordan.slingerland@gmail.com> wrote:

Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it.  I have specifically asked what stipulation.  In a VDI environment with linked clones, etc.  Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.

I have A LOT of trouble believing that.  So, 4:1 dedup guaranteed or what? 

Any comments welcome.

--Jordan

______________________________ _________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/ mailman/listinfo/toasters




______________________________ _________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/ mailman/listinfo/toasters




______________________________ _________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/ mailman/listinfo/toasters



______________________________ _________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/ mailman/listinfo/toasters


______________________________ _________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/ mailman/listinfo/toasters



_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
RE: AFF and 4:1 guaranteed efficiency [ In reply to ]
I just want to point out a few things based on my experience and feedback from other people.



Pure Storage - block only. You do not get any bells and whistles.

XIO – everyone that owns one has something bad to say about it

TinTri – NFS only, each node is a separate unit, VM aware.. has a place for certain markets

SolidFire – amazing potential, you can have disk failures or a complete node fail and stay up and running , but again, block only and no bells and whistles.

Netapp AFF – all protocols supported, the entire snap manager suite (exchange, SQL and sharepoint are absolutely outstanding) and with their 4:1 guarantee they will give you more shelves if they don’t hit that mark. They also give you controller upgrades after 3 years.



In my experience the “bake offs” have been between XIO, Pure Storage and Netapp on critical EMR systems like EPIC and McKesson. Netapp’s AFF has always been the one that came out on top as far as performance. EMC has been the one willing to give their product away just to keep their foot print.



With all that being said, you need to figure out what fits your needs. I have customers that want to boot UCS blades from SAN, Run NFS datastores and migrate their windows file servers to native CIF’s shares on their storage system. The only storage company that can do all of that is Netapp (I won’t bring up Isolation because their block size is terrible with small files)



So, if you are only worried about the 4:1 efficiencies who cares? They will give you more disks and you still get the Cadillac.





From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of tmac
Sent: Friday, September 23, 2016 10:34 AM
To: jordan slingerland
Cc: toasters@teaparty.net
Subject: Re: AFF and 4:1 guaranteed efficiency



I vaguely remember running a test on an ONTAP 9 simulator.



What I found was that if the destination aggr had compaction enabled, then snapmirror into that aggr would also have compacted data.

This DOES NOT work with XDP (snapvault & version-flexible snapmirror)








--tmac



Tim McCarthy, Principal Consultant

Proud Member of the #NetAppATeam <https://twitter.com/NetAppATeam>

I Blog at <https://tmacsrack.wordpress.com/> TMACsRack


Image removed by sender. NetApp - In partnership with Alpine Testing Solutions <https://www.certmetrics.com/netapp/public/badge.aspx?t=c&d=2012-11-05&i=35&ci=NETAPP00041276> Image removed by sender. NetApp Certified Data Administrator, ONTAP <https://www.certmetrics.com/netapp/public/badge.aspx?t=c&d=2012-11-08&i=36&ci=NETAPP00041276> Image removed by sender. NetApp Certified Implementation Engineer - SAN Specialist, ONTAP <https://www.certmetrics.com/netapp/public/badge.aspx?t=c&d=2015-10-13&i=38&ci=NETAPP00041276> Image removed by sender. NetApp Certified Storage Installation Engineer, ONTAP <https://www.certmetrics.com/netapp/public/badge.aspx?t=c&d=2015-10-15&i=11&ci=NETAPP00041276> Image removed by sender. NetApp Certified Implementation Engineer - Data Protection SpecialistImage removed by sender.Image removed by sender.





NetApp Candidate ID: NETAPP00041276

FlexPod Design: Oct 2015 - Jan 2018, S0N62WE1BMVEYF3M

FlexPod Implementation: Oct 2015 - Jan 2018, JH3QJT4KLEQ41HPH



RHCE6 <https://www.redhat.com/wapps/training/certification/verify.html?certNumber=110-107-141&isSearch=False&verify=Verify> 110-107-141



On Fri, Sep 23, 2016 at 11:25 AM, jordan slingerland <jordan.slingerland@gmail.com> wrote:

My understanding is that both are block based operations so it makes sense to me that the blocks would be put down on disk unchanged. Perhaps it has something to do with the fact that you may want to reverse the snapmirror back to the system that presumably does not have compaction enabled or support the feature, so the blocks are left uncompacted? If the vol move is within the same controller AFF the lack of backward compatibility is not a concern. Just a though, I don't know.



On Fri, Sep 23, 2016 at 10:57 AM, Francis Kim <fkim@berkcom.com> wrote:

Strange how compaction appears to work with vol move but not with snap mirror.



.


On Sep 23, 2016, at 4:39 AM, Steiner, Jeffrey <Jeffrey.Steiner@netapp.com> wrote:

A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move.



Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import.

Sent from my mobile phone.


On 23 Sep 2016, at 02:10, Jeffrey Mohler <jmohler@yahoo-inc.com> wrote:

Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else. (SSET diagnosis)

However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.

It must transfer, as far as we're told today, via a host/file based migration. Be thinking bout this when you consider an AFF migration. It must be done outside on ONTAP.



We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.





_________________________________

Jeff Mohler <mailto:jmohler@yahoo-inc.com>

Tech Yahoo, Storage Architect, Principal

(831)454-6712
YPAC Gold Member

Twitter: @PrincipalYahoo
CorpIM: Hipchat & Iris





On Friday, September 23, 2016 1:53 AM, Mike Gossett <cmgossett@gmail.com> wrote:



my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.



The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.







On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland <jordan.slingerland@gmail.com> wrote:

Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.

I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?

Any comments welcome.

--Jordan


______________________________ _________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/ mailman/listinfo/toasters <http://www.teaparty.net/mailman/listinfo/toasters>





_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters



_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters




_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
RE: AFF and 4:1 guaranteed efficiency [ In reply to ]
Okay, I heard back. Engineering says that snapmirror and vol-move should BOTH trigger compaction. If you didn't see savings, perhaps something about thin provisioning was not configured. The savings could have been there, they just weren't visible if the volume settings weren't correct.

From: Steiner, Jeffrey
Sent: Friday, September 23, 2016 8:29 AM
To: 'jordan slingerland' <jordan.slingerland@gmail.com>; NGC-fkim-berkcom.com <fkim@berkcom.com>
Cc: toasters@teaparty.net
Subject: RE: AFF and 4:1 guaranteed efficiency

I would guess that snapmirror is currently looking at the physical blocks on disk, whereas a vol-move is a step up the chain and it can look at block that are actually a block within a block.

I'll check with engineering and report back...

From: jordan slingerland [mailto:jordan.slingerland@gmail.com]
Sent: Friday, September 23, 2016 10:26 AM
To: NGC-fkim-berkcom.com <fkim@berkcom.com<mailto:fkim@berkcom.com>>
Cc: Steiner, Jeffrey <Jeffrey.Steiner@netapp.com<mailto:Jeffrey.Steiner@netapp.com>>; toasters@teaparty.net<mailto:toasters@teaparty.net>
Subject: Re: AFF and 4:1 guaranteed efficiency

My understanding is that both are block based operations so it makes sense to me that the blocks would be put down on disk unchanged. Perhaps it has something to do with the fact that you may want to reverse the snapmirror back to the system that presumably does not have compaction enabled or support the feature, so the blocks are left uncompacted? If the vol move is within the same controller AFF the lack of backward compatibility is not a concern. Just a though, I don't know.

On Fri, Sep 23, 2016 at 10:57 AM, Francis Kim <fkim@berkcom.com<mailto:fkim@berkcom.com>> wrote:
Strange how compaction appears to work with vol move but not with snap mirror.

.

On Sep 23, 2016, at 4:39 AM, Steiner, Jeffrey <Jeffrey.Steiner@netapp.com<mailto:Jeffrey.Steiner@netapp.com>> wrote:
A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move.

Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import.

Sent from my mobile phone.

On 23 Sep 2016, at 02:10, Jeffrey Mohler <jmohler@yahoo-inc.com<mailto:jmohler@yahoo-inc.com>> wrote:
Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else. (SSET diagnosis)

However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.
It must transfer, as far as we're told today, via a host/file based migration. Be thinking bout this when you consider an AFF migration. It must be done outside on ONTAP.

We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.


_________________________________
Jeff Mohler<mailto:jmohler@yahoo-inc.com>
Tech Yahoo, Storage Architect, Principal
(831)454-6712
YPAC Gold Member
Twitter: @PrincipalYahoo
CorpIM: Hipchat & Iris


On Friday, September 23, 2016 1:53 AM, Mike Gossett <cmgossett@gmail.com<mailto:cmgossett@gmail.com>> wrote:

my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.

The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.



On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland <jordan.slingerland@gmail.com<mailto:jordan.slingerland@gmail.com>> wrote:
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.
I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?
Any comments welcome.
--Jordan

______________________________ _________________
Toasters mailing list
Toasters@teaparty.net<mailto:Toasters@teaparty.net>
http://www.teaparty.net/ mailman/listinfo/toasters<http://www.teaparty.net/mailman/listinfo/toasters>


_______________________________________________
Toasters mailing list
Toasters@teaparty.net<mailto:Toasters@teaparty.net>
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters@teaparty.net<mailto:Toasters@teaparty.net>
http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________
Toasters mailing list
Toasters@teaparty.net<mailto:Toasters@teaparty.net>
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters@teaparty.net<mailto:Toasters@teaparty.net>
http://www.teaparty.net/mailman/listinfo/toasters

1 2  View All