ADSM-L

Re: Compression Ratio ?

2002-01-23 03:13:49
Subject: Re: Compression Ratio ?
From: Reinhard Mersch <mersch AT UNI-MUENSTER DOT DE>
Date: Wed, 23 Jan 2002 09:11:05 +0100
Hi,

to give you a clue: based in Andy's suggestion, I just did the
following:

dsmadmc -id=operator -pa=xxx q act begind=-10 msgno=4968 |
grep "compressed by" |
awk 'BEGIN{n=0;s=0;}{gsub("%","",$3);if($3){n+=1;s+=$3;}}END{print s/n;}'

The result was "50.5865", so our clients compress their data by
app. 50% on the average. The number of sessions contributing to this
result (i.e. not having 0% compression) was 1323.

This is a typical university environment, i.e. lots of workstations
and server, but no databases backed up.

> If there is an "official" number, I do not know what it is. My
> personal opinion (thus not necessarily that of IBM): practically
> speaking, such numbers are nearly worthless because of the
> variances in actual ratios.

> Just for the heck of it, I visited the sites of a couple of
> well-known and popular compression products, and I didn't see
> any hard numbers from them, either. I suspect that this may be
> due to the same reason: because different kinds of data have
> different compression characteristics.

> You would probably get a more meaningful number by examining the
> results you see on your own site. Check the TSM server for
> message ANE4968I:

>    ANE4968I (Session: ssss, Node: xxxxxxxx)  Objects
>             compressed by:                    n%

> You can probably discard cases where the percentage is 0, as the
> client is probably not using compression.

> Regards,

> Andy

> Andy Raibeck
> IBM Software Group
> Tivoli Storage Manager Client Development
> Internal Notes e-mail: Andrew Raibeck/Tucson/IBM@IBMUS
> Internet e-mail: storman AT us.ibm DOT com

> The only dumb question is the one that goes unasked.
> The command line is your friend.
> "Good enough" is the enemy of excellence.




> Othonas Xixis/Durham/IBM@ibmus
> Sent by: "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU>
> 01/22/2002 07:37
> Please respond to "ADSM: Dist Stor Manager"


>         To:     ADSM-L AT VM.MARIST DOT EDU
>         cc:
>         Subject:        Re: Compression Ratio ?



> Andy,
> what is the "official" TSM Client (4.2.n +) compression ratio ?
> I know that it can vary depending of the type of the data, however there
> should be an "announcable" number that admins can use to project.

> Thanks in advance for yr response.

> Brgds,

> Othonas






> Andrew Raibeck/Tucson/IBM@IBMUS AT VM.MARIST DOT EDU> on 01/22/2002 09:12:17 
> AM

> Please respond to "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU>

> Sent by:    "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU>


> To:    ADSM-L AT VM.MARIST DOT EDU
> cc:
> Subject:


> I forgot to mention:

> Starting with the 4.2 client, you can now use include.compression and
> exclude.compression client options. For example:

>    exclude.compression *:\...\*.zip

> will prevent any *.zip file from being compressed, even if COMPRESSION YES
> and COMPRESSALWAYS are in effect.

> However, the 4.2 client is not supported with the 3.1 server.

> Regards,

> Andy

> Andy Raibeck
> IBM Software Group
> Tivoli Storage Manager Client Development
> Internal Notes e-mail: Andrew Raibeck/Tucson/IBM@IBMUS
> Internet e-mail: storman AT us.ibm DOT com

> The only dumb question is the one that goes unasked.
> The command line is your friend.
> "Good enough" is the enemy of excellence.
> ----- Forwarded by Andrew Raibeck/Tucson/IBM on 01/22/2002 07:10 -----


> Andrew Raibeck
> 01/22/2002 07:04


>         To:     "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU>
>         cc:
>         From:   Andrew Raibeck/Tucson/IBM@IBMUS
>         Subject:        Re:






> ADSM and TSM do not "know" that a file is compressed, per se. However, it
> does detect if the file is growing during compression (which is most
> likely because the file is already compressed). If the file grows during
> compression and COMPRESSALWAYS YES (the default) is in effect, then it
> will keep compressing the file anyway. This is causing the negative
> compression that you are seeing. If COMPRESSALWAYS NO is in effect and the
> file grows during compression, then all files in the current transaction
> will be resent to the server, but the file that grew wll be sent
> uncompressed. The trade-off is that in resending all the other files in
> the current transaction, you may see a performance degradation in over-all
> elapsed time, especially if there are lots of files that grow during
> compression.

> Regard,

> Andy

> Andy Raibeck
> IBM Software Group
> Tivoli Storage Manager Client Development
> Internal Notes e-mail: Andrew Raibeck/Tucson/IBM@IBMUS
> Internet e-mail: storman AT us.ibm DOT com

> The only dumb question is the one that goes unasked.
> The command line is your friend.
> "Good enough" is the enemy of excellence.




> Wolfgang Rest <webmaster AT HACKENSCHMIEDE DOT COM>
> Sent by: "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU>
> 01/22/2002 06:04
> Please respond to "ADSM: Dist Stor Manager"


>         To:     ADSM-L AT VM.MARIST DOT EDU
>         cc:
>         Subject:



> 01/21/2002 21:41:55 ANE4952I (Session: 196, Node: APOLLO)  Total number of
> objects inspected:  129,216
> 01/21/2002 21:41:55 ANE4954I (Session: 196, Node: APOLLO)  Total number of
> objects backed up:   43,830
> 01/21/2002 21:41:55 ANE4958I (Session: 196, Node: APOLLO)  Total number of
> objects updated:          0
> 01/21/2002 21:41:55 ANE4960I (Session: 196, Node: APOLLO)  Total number of
> objects rebound:          0
> 01/21/2002 21:41:55 ANE4957I (Session: 196, Node: APOLLO)  Total number of
> objects deleted:          0
> 01/21/2002 21:41:55 ANE4959I (Session: 196, Node: APOLLO)  Total number of
> objects failed:           4
> 01/21/2002 21:41:55 ANE4961I (Session: 196, Node: APOLLO)  Total number of
> bytes transferred:     5.11 GB
> 01/21/2002 21:41:55 ANE4963I (Session: 196, Node: APOLLO)  Data transfer
> time:                  911.71 sec
> 01/21/2002 21:41:55 ANE4966I (Session: 196, Node: APOLLO)  Network data
> transfer rate:        5,886.47 KB/sec
> 01/21/2002 21:41:55 ANE4967I (Session: 196, Node: APOLLO)  Aggregate data
> transfer rate:        992.76 KB/sec
> 01/21/2002 21:41:55 ANE4968I (Session: 196, Node: APOLLO)  Objects
> compressed by:                  -16%
> 01/21/2002 21:41:55 ANE4964I (Session: 196, Node: APOLLO)  Elapsed
> processing time:            01:30:05


> Objects compressed by: -16%  ?!?!?!

> How can this be? There are many .zip Files on this Server.. but i thought
> ADSM will check if a File is
> already compressed and dont compress it again.

> I am running on Client side 3.1.0.8 and Server 3.1.2.90

> best regards

--
Reinhard Mersch                        Westfaelische Wilhelms-Universitaet
Reinhard Mersch                        Westfaelische Wilhelms-Universitaet
Zentrum fuer Informationsverarbeitung - ehemals Universitaetsrechenzentrum
Roentgenstrasse 9-13, D-48149 Muenster, Germany      Tel: +49(251)83-31583
E-Mail: mersch AT uni-muenster DOT de                       Fax: 
+49(251)83-31653
<Prev in Thread] Current Thread [Next in Thread>