We often use IBM Storwize systems, V3700 for smaller setups and V5020 for
larger configurations.
>From 24 up to 60 4-6TB nearline drives all in RAID 6.
2x8 core Lenovo servers with internal SSD's in RAID 5 attached to (super
fast) internal raid controllers for database and active log reaching 140K
iop/s in IBM Spectrum Protect database benchmarks.
We outfit them with 192GB of memory when using replication.
These systems do 800MB/s inline deduplication and compression without CPU
room to spare in some load testing. I have seen that 10Gb/s line filled up
many times and even use LACP configurations with two 10Gb/s adapters to
make sure I'm not holding it back.
We usually run Linux, SLES to be exact.
The first time I saw one of these things do inline dedup at those
speeds...the inner nerd got a little emotional. :-)
We've really got some amazing tech right here with these containerpools.
On Wed, Aug 24, 2016 at 1:08 PM, Rhodes, Richard L. <
rrhodes AT firstenergycorp DOT com> wrote:
> I'm curious, what kind of storage system do ya'll use for your container
> pools?
>
>
>
> -----Original Message-----
> From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf
> Of
> David Ehresman
> Sent: Tuesday, August 23, 2016 8:56 AM
> To: ADSM-L AT VM.MARIST DOT EDU
> Subject: *EXTERNAL* Re: deleing data from a containerpool
>
> We are getting a 78% savings on a mixed workload (BA, Oracle, VM,
> Exchange, SQL) with about 350 nodes moving around 10TB of data a night.
>
> David
>
> -----Original Message-----
> From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf
> Of
> Stefan Folkerts
> Sent: Tuesday, August 23, 2016 8:43 AM
> To: ADSM-L AT VM.MARIST DOT EDU
> Subject: Re: [ADSM-L] deleing data from a containerpool
>
> This inline deduplication and compression thing really is amazing;
>
> excerpt from q stgpool f=d
>
> Deduplication Savings: 153,840 G (90.83%)
> Compression Savings: 8,711 G (56.07%)
> Total Space Saved: 162,550 G (95.97%)
>
> So on top of the 90.83% deduplication saving (that dont' require any
> reclaiming) this customer saves another 56.07% of the stored data due to
> compression making it a total of 95.97% space saved.
> These figures require a lot of duplicate data of course but still,
> containerpools FTW!
> We call it the "cool pool" btw. :-)
>
>
>
> On Mon, Aug 22, 2016 at 4:01 PM, Del Hoobler <hoobler AT us.ibm DOT com>
> wrote:
>
> > Minor correction.
> >
> > Inline compression for container pools was added in March in 7.1.5.
> >
> >
> > IBM Spectrum Protect 7.1.5 - Inline compression:
> > - Performed in-line after deduplication to provide additional storage
> > savings
> > - Negligible impact on resources – uses latest and most efficient
> > compression algorithms
> > - Can potentially double (or more) your storage savings after
> > deduplication
> >
> >
> > Del
> >
> > ----------------------------------------------------
> >
> >
> > "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU> wrote on 08/22/2016
> > 09:53:08 AM:
> >
> > > From: "Loon, Eric van (ITOPT3) - KLM" <Eric-van.Loon AT KLM DOT COM>
> > > To: ADSM-L AT VM.MARIST DOT EDU
> > > Date: 08/22/2016 09:53 AM
> > > Subject: Re: deleing data from a containerpool
> > > Sent by: "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU>
> > >
> > > Indeed, TSM 7.1.0 to 7.1.5 only supported deduplication, additional
> > > compression was introduced in 7.1.6.
> > > Kind regards,
> > > Eric van Loon
> > > Air France/KLM Storage Engineering
> > >
> > > -----Original Message-----
> > > From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On
> > > Behalf Of David Ehresman
> > > Sent: maandag 22 augustus 2016 15:12
> > > To: ADSM-L AT VM.MARIST DOT EDU
> > > Subject: Re: deleing data from a containerpool
> > >
> > > At the most recent levels of TSM, it both dedups and compresses but
> > > make sure you are at a level that does both. There was a level that
> > > only did dedup but not compression.
> > >
> > > David
> > >
> > > -----Original Message-----
> > > From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On
> > > Behalf Of Rhodes, Richard L.
> > > Sent: Monday, August 22, 2016 9:07 AM
> > > To: ADSM-L AT VM.MARIST DOT EDU
> > > Subject: Re: [ADSM-L] deleing data from a containerpool
> > >
> > > >But I totally agree, everyone who is using file device
> > >
> > > >classes or expensive backend deduplication (like Data Domain or
> > Protectier)
> > >
> > > >should seriously consider switching to container pools.
> > >
> > >
> > >
> > >
> > >
> > > We currently use DataDomains.
> > >
> > > With a DD it dedups what it can, then compresses the rest.
> > >
> > > Does TSM also try and compress what is leftover after dedup?
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > -----Original Message-----
> > >
> > > From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On
> > > Behalf Of Loon, Eric van (ITOPT3) - KLM
> > >
> > > Sent: Tuesday, August 16, 2016 3:39 AM
> > >
> > > To: ADSM-L AT VM.MARIST DOT EDU
> > >
> > > Subject: Re: deleing data from a containerpool
> > >
> > >
> > >
> > > Hi Stefan!
> > >
> > > Our database is on SSD in an IBM V3700, but the time needed for a
> > > del filespace can be significant though. But I totally agree,
> > > everyone who is using file device classes or expensive backend
> > > deduplication (like Data Domain or Protectier) should seriously
> > > consider switching to container pools. We are working on a design
> > > for our next TSM servers and we are able to lower our costs per TB
> > > by 75% compared to the old design based on the Data Domain!
> > >
> > > Kind regards,
> > >
> > > Eric van Loon
> > >
> > > Air France/KLM Storage Engineering
> > >
> > >
> > >
> > > -----Original Message-----
> > >
> > > From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On
> > > Behalf Of Stefan Folkerts
> > >
> > > Sent: dinsdag 16 augustus 2016 8:33
> > >
> > > To: ADSM-L AT VM.MARIST DOT EDU
> > >
> > > Subject: Re: deleing data from a containerpool
> > >
> > >
> > >
> > > Yes, I too have noticed this and it is something to keep in mind.
> > >
> > > At the same time, I think almost everybody using this pool will be
> > > using SSD's for the database the impact will be overseeable.
> > >
> > > But the directory containerpool is still the best thing to happen to
> > > Spectrum Protect since replication came along if you ask me. great
> > > performance increase over fileclass restores, no more stopping
> > > reclaims during the day to increase restore performance, no more
> > > messing with numopenvolsallowed and reclaim values and number of
> > > processes to optimize daily operations and restore speed...oh, and
> > > compression that saves an easy 30-50% storage and license cost on
> > > top of the deduplication!
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Mon, Aug 15, 2016 at 11:20 AM, Loon, Eric van (ITOPT3) - KLM <
> > > Eric-van.Loon AT klm DOT com> wrote:
> > >
> > >
> > >
> > > > Hi all!
> > >
> > > > After doing some extensive testing with a directory container
> > >
> > > > storagepool I noticed a significant change compared to the old
> > >
> > > > traditional storage pools.
> > >
> > > > In a traditional storage pool TSM stores a file like an object. In
> > >
> > > > most cases one file is one object as far as I could see. Deleting
> this
> >
> > >
> > > > data is very fast: a delete filespace runs rather fast because TSM
> > >
> > > > only has to delete the objects. So deleting a large database client
> > >
> > > > with multiple TBs takes a few seconds or maybe a few minutes.
> > >
> > > > When you are using a container storage pool everything changes. Files
> > >
> > > > are still stored as objects, but objects are split into chunks. The
> > >
> > > > average size of a chuck is approx. 100 KB and TSM performs dedup on
> > >
> > > > this chuck level. So if you now delete a large file, TSM has to
> > >
> > > > inspect every chunk to see if it is unique or not. When it is unique
> > >
> > > > it will be deleted, otherwise not. If you delete a file which is for
> > >
> > > > instance 40 GB in size, TSM has to inspect around 420,000 chucks
> > >
> > > > before the object can be deleted. I noticed that this takes several
> > >
> > > > seconds to complete, so one has to take into consideration that the
> > >
> > > > deletion of large clients requires significant more time to
> > > complete than one is used to.
> > >
> > > > Deleing a client with little more than 1 TB of Oracle data was
> running
> >
> > >
> > > > for more than 20 minutes. So a delete filespace for really large
> > >
> > > > database clients can run for hours! Has anyone else noticed this
> > > behavior too?
> > >
> > > > Kind regards,
> > >
> > > > Eric van Loon
> > >
> > > > Air France/KLM Storage Engineering
> > >
> > > > ********************************************************
> > >
> > > > For information, services and offers, please visit our web site:
> > >
> > > > https://urldefense.proofpoint.com/v2/url?
> > >
> > u=http-3A__www.klm.com&d=AwIGaQ&c=SgMrq23dbjbGX6e0ZsSHgEZX6A4IAf
> > 1SO3AJ2bNrHlk&r=dOGCMY197NTNH1k_wcsrWS3_fxedKW4rpKJ8cHCD2L8&m=
> > vUuUnchIk8qp8ANX9ecD5HSZje8iCRgiNUPhmahQWTQ&s=
> > eE7XROkp9Iv02y6CJDM84muZgTbKNhpt7nAgYPrCs-0&e=
> > > . This e-mail and any attachment may contain
> > >
> > > > confidential and privileged material intended for the addressee only.
> > >
> > > > If you are not the addressee, you are notified that no part of the
> > >
> > > > e-mail or any attachment may be disclosed, copied or distributed, and
> > >
> > > > that any other action related to this e-mail or attachment is
> strictly
> >
> > >
> > > > prohibited, and may be unlawful. If you have received this e-mail by
> > >
> > > > error, please notify the sender immediately by return e-mail, and
> > > delete this message.
> > >
> > > >
> > >
> > > > Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or
> > >
> > > > its employees shall not be liable for the incorrect or incomplete
> > >
> > > > transmission of this e-mail or any attachments, nor responsible
> > > for any delay in receipt.
> > >
> > > > Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal
> > >
> > > > Dutch
> > >
> > > > Airlines) is registered in Amstelveen, The Netherlands, with
> > >
> > > > registered number 33014286
> > >
> > > > ********************************************************
> > >
> > > >
> > >
> > > ********************************************************
> > >
> > > For information, services and offers, please visit our web site:
> > > https://urldefense.proofpoint.com/v2/url?
> > >
> > u=http-3A__www.klm.com&d=AwIGaQ&c=SgMrq23dbjbGX6e0ZsSHgEZX6A4IAf
> > 1SO3AJ2bNrHlk&r=dOGCMY197NTNH1k_wcsrWS3_fxedKW4rpKJ8cHCD2L8&m=
> > vUuUnchIk8qp8ANX9ecD5HSZje8iCRgiNUPhmahQWTQ&s=
> > eE7XROkp9Iv02y6CJDM84muZgTbKNhpt7nAgYPrCs-0&e=
> > > . This e-mail and any attachment may contain confidential and
> > > privileged material intended for the addressee only. If you are not
> > > the addressee, you are notified that no part of the e-mail or any
> > > attachment may be disclosed, copied or distributed, and that any
> > > other action related to this e-mail or attachment is strictly
> > > prohibited, and may be unlawful. If you have received this e-mail by
> > > error, please notify the sender immediately by return e-mail, and
> > > delete this message.
> > >
> > >
> > >
> > > Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/
> > > or its employees shall not be liable for the incorrect or incomplete
> > > transmission of this e-mail or any attachments, nor responsible for
> > > any delay in receipt.
> > >
> > > Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal
> > > Dutch Airlines) is registered in Amstelveen, The Netherlands, with
> > > registered number 33014286
> > >
> > > ********************************************************
> > >
> > >
> > >
> > > ********************************************************
> > > For information, services and offers, please visit our web site:
> > > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.klm.
> com&d=AwIFaQ&c=SgMrq23dbjbGX6e0ZsSHgEZX6A4IAf1SO3AJ2bNrHlk&r=
> dOGCMY197NTNH1k_wcsrWS3_fxedKW4rpKJ8cHCD2L8&m=
> 2khjm5kIUk4HAMhbAmSErW4psymX49EvvcPlonm458c&s=KERJhiJfWrogTNbgmZOr6_
> HGAgfaXIQntXRwL231Di4&e= . This e-mail and any attachment may contain
> > > confidential and privileged material intended for the addressee
> > > only. If you are not the addressee, you are notified that no part of
> > > the e-mail or any attachment may be disclosed, copied or
> > > distributed, and that any other action related to this e-mail or
> > > attachment is strictly prohibited, and may be unlawful. If you have
> > > received this e-mail by error, please notify the sender immediately
> > > by return e-mail, and delete this message.
> > >
> > > Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/
> > > or its employees shall not be liable for the incorrect or incomplete
> > > transmission of this e-mail or any attachments, nor responsible for
> > > any delay in receipt.
> > > Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal
> > > Dutch Airlines) is registered in Amstelveen, The Netherlands, with
> > > registered number 33014286
> > > ********************************************************
> > >
> >
> >
> >
>
|
