ADSM-L

Re: [ADSM-L] Ramifications of turning client compression on?

2007-05-02 13:11:51
Subject: Re: [ADSM-L] Ramifications of turning client compression on?
From: Kelly Lipp <lipp AT STORSERVER DOT COM>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Wed, 2 May 2007 11:11:07 -0600
Kudos to John Schneider for his cogent presentation for solving a
specific problem.  I concur with everything he said and will incorporate
it into my newly revised response to this.  Since his concern was not
backup time, but rather efficiencies throughout the system, his points
are quite valid.

Some of the daily processing load can be minimized too by using the
copystgpool option on a primary pool.  This only, however, for clients
moving a lot of data.  I would not suggest this notion for lots of
smaller clients.  Way too many tape mounts.

The optimization of daily processing is often very critical especially
if hardware resources are scarce.  I sense a Whitepaper on this... 


Kelly J. Lipp
VP Manufacturing & CTO
STORServer, Inc.
485-B Elkton Drive
Colorado Springs, CO 80907
719-266-8777
lipp AT storserver DOT com

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf Of
Andrew Raibeck
Sent: Tuesday, May 01, 2007 7:06 PM
To: ADSM-L AT VM.MARIST DOT EDU
Subject: Re: [ADSM-L] Ramifications of turning client compression on?

Something else to be aware of when using client compression:

When client compression is used, tape volume utilization might appear
low.
This is because the data is already compressed by the time it gets to
tape. Hence the tape utilization will probably appear closer to the
tape's "native" capacity rather than the higher capacities that can be
achieved through the tape drive's own hardware compression capabilities.
The reason for this is that compressed data does not recompress very
well.

For example, a tape whose native capacity is 100 GB, but has an
advertised capacity of, say, 200 - 300 GB, will probably show a
utilization closer to 100 GB.

This is not a problem, but is a frequent newbie question ("Why is TSM
not utilizing the full capacity of my tapes?").

Regards,

Andy

Andy Raibeck
IBM Software Group
Tivoli Storage Manager Client Product Development Level 3 Team Lead
Internal Notes e-mail: Andrew Raibeck/Tucson/IBM@IBMUS Internet e-mail:
storman AT us.ibm DOT com

IBM Tivoli Storage Manager support web page:
http://www.ibm.com/software/sysmgmt/products/support/IBMTivoliStorageMan
ager.html

The only dumb question is the one that goes unasked.
The command line is your friend.
"Good enough" is the enemy of excellence.

"ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU> wrote on 2007-05-01
17:50:31:

> My first thought after reading this goes back to what the main goal
should
> be in any TSM environment:  How are your restores affected?  Will you 
> still be able to meet your SLAs for restore times (especially a full 
> restore)?  That is an area that is often overlooked until it is too
late.
> IMO, too much engineering time in many TSM environments is spent 
> around backups, when a bulk of the time should be spent around 
> engineering the fastest and most reliable restores possible.  Just 
> something to remember when considering the benefits/drawbacks of
client compression.
>
> ______________________________
>
> John Monahan
> Consultant
> Logicalis, Inc.
> 5500 Wayzata Blvd Suite 315
> Golden Valley, MN 55416
> Office: 763-417-0552 x109
> Mobile: 952-221-6938
> Fax:  763-417-0554
> John.Monahan AT us.logicalis DOT com
> http://www.us.logicalis.com
>
>
>
>
> "Schneider, John" <schnjd AT STLO.MERCY DOT NET> Sent by: "ADSM: Dist Stor 
> Manager" <ADSM-L AT VM.MARIST DOT EDU>
> 05/01/2007 05:18 PM
> Please respond to
> "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU>
>
>
> To
> ADSM-L AT VM.MARIST DOT EDU
> cc
>
> Subject
> Re: Ramifications of turning client compression on?
>
>
>
>
>
>
> I am going to weigh in on this one.  Client compression can work 
> great, depending on the kind of problems you are trying to solve, and 
> if you do it right.  Here was our situation:
>
> I recently took on responsibility for a TSM environment with 1 shared 
> library with 14 tape drives, 6 TSM AIX servers, and about 760 clients 
> backed up per night.
>
> When I started working here, we were at the very edge of keeping up.
> Disk storage pools were filling up in the middle of the night so much 
> that migration was running for hours in the middle of backups, and 
> backup storage pools weren't completing in the timeframe we had during

> the day.  So we were forced to cancel backup storage pools on some 
> days just to get the offsite tapes off before 5PM.  Then migration 
> would run most of the time until the night's backups began.  We only 
> had a couple hours per day to run reclamation, because  migration 
> would need to kick off and use the tape drives.  Things were a mess.
>
> I gradually rolled in client compression.  Today we are backing up 
> over 100 clients more than before, and about 1.5 TB per day more data.
> Migration never happens in the middle of the night (except for an 
> extraordinary circumstance).  Backup stgpools complete every day, and 
> are finished by about noon.  Migration runs until early evening, and 
> we have 6-7 hours to run reclamation.
>
> While part of the improvement came from scheduling changes to keep all

> the tape drives busier, I credit client compression with a big part of

> it.  The TSM servers are only having to absorb and write to disk about

> half as much data.  So the network load is cut in half, the data to 
> migrate is cut in half, backup stgpools are cut in half, and so on.
>
> As you know, a TSM server has to read and write the same data lots of 
> times during the life of the data.  Migration, backup stgpool, and who

> knows how many iterations of reclamation before the data finally 
> expires will mean that the data has to be copied from disk or tape 
> multiple times.  When you are using compression on the tape drive, 
> each time you have to copy the data, it is uncompressed, copied in 
> it's original form into TSM's memory, and back out trough the FC 
> switches, where it is recompressed back onto tape.  But during the 
> time it is going through the SAN, and TSM's memory and disk, it is in 
> uncompressed form, and that means more load and slower performance.
>
> With client compression, the data is compressed once, and stays in 
> it's most efficient size for the entire time TSM has to handle it, no 
> matter how many times TSM has to copy it.  I think that constitutes a 
> significant improvement in efficiency.
>
> But everybody asks, "Don't the clients take a lot longer to back up?"
> Each individual client takes longer to back up, but you can compensate

> for that by starting more backups earlier in the evening.  Since the 
> clients are only going to send about half as much data, you can have 
> twice as many simultaneous clients running without hurting anything.  
> We found that most incremental backups ran about 50% longer.  That 
> sounds bad, but most of our client backups ran in 1-2 hours.  So 
> instead of starting them at 2am for instance, we start them at 
> midnight instead, and they finish at the same time as before.  What is
the harm in that?
> Even the clients that take 4 hours might take only 5-6, but as long as

> we can start them early enough to all get done within the window, that

> is no problem.
>
> The only exceptions to this rule are very large clients that might 
> take
> 10-12 hours already, or clients that are significantly CPU 
> constrained, and don't have any CPU to throw at doing the compression.

> But those turned out to be a tiny fraction of the clients in our
environment.
> Even our older and slower Windows servers were able to use client 
> compression with no impact to the production application.
>
> Best Regards,
>
> John D. Schneider
> Sr. System Administrator - Storage
> Sisters of Mercy Health System
> 3637 South Geyer Road
> St. Louis, MO.  63127
> Email:  schnjd AT stlo.mercy DOT net
> Office: 314-364-3150, Cell:  314-486-2359
>
>
> -----Original Message-----
> From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf 
> Of Kelly Lipp
> Sent: Tuesday, May 01, 2007 2:10 PM
> To: ADSM-L AT VM.MARIST DOT EDU
> Subject: Re: [ADSM-L] Ramifications of turning client compression on?
>
>
> The first question I would ask is why are you doing this?  If you hope

> to reduce the amount of TIME it takes to do a backup, you won't.  If 
> you hope to reduce the amount of DISK SPACE in your disk pools, you 
> will, but at what cost.
>
> In almost all cases (I don't say all because words like all or never 
> annoy me) compression is best done in hardware rather than software.  
> So let your tape technology do the compression.
>
> That said, in an all disk based TSM environment it may well make sense

> to let clients do compression to reduce the amount of disk required.
> Personally, I'm hoping for some clever SAN vendor to put compression 
> into their controllers so the hardware will do it for me.
>
> Thanks,
>
>
> Kelly J. Lipp
> VP Manufacturing & CTO
> STORServer, Inc.
> 485-B Elkton Drive
> Colorado Springs, CO 80907
> 719-266-8777
> lipp AT storserver DOT com
>
> -----Original Message-----
> From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf 
> Of Joni Moyer
> Sent: Tuesday, May 01, 2007 12:44 PM
> To: ADSM-L AT VM.MARIST DOT EDU
> Subject: [ADSM-L] Ramifications of turning client compression on?
>
> Hello All!
>
> I am considering turning client compression on for our Lotus Notes 
> Domino and Oracle servers.  What should be looked at/considered before

> client compression is turned on?  How do you monitor the impact of 
> turning client compression on?  Any ideas/suggestions are appreciated!
> Thanks!
>
> ********************************
> Joni Moyer
> Highmark
> Storage Systems, Storage Mngt Analyst III Phone Number: (717)302-9966
> Fax: (717) 302-9826
> joni.moyer AT highmark DOT com
> ********************************