Client Data Compression

From: Peter Thomas on 01/31/97 05:14 PM
In the process of testing out the functionality of ADSM from Windows/NT,
Netware, UNIX and OS/2 to a UNIX based server, we've created an artificial
number of data files and directories we've observed that some alarming
trends if ADSM Compression is enabled.



The directory structure in question contaions roughly 1 GB of data in
35,000 files. If the client disables ADSM compression then both the
Accounting Log and end of job summary indicates that 975 MB of data has
been transferred in 35,000 objects. However, if compression is enabled, the
data transferred volume increases to 2.2 GB and number of objects to
roughly 85,000.



Looking at the log one sees a large number of entries of the form



Normal File-->               4 /adsmtest/dickies/.rhosts  Sent
Normal File-->             130 /adsmtest/dickies/.sh_history  Sent
Directory-->            34,816 /adsmtest/dickies/big  Sent
Normal File-->          35,712 /adsmtest/dickies/big/F10008  Compressed
Data Grew
Retry # 1  Directory-->               512 /adsmtest/dickies  Sent
Retry # 1  Normal File-->               4 /adsmtest/dickies/.rhosts  Sent
Retry # 1  Normal File-->             130 /adsmtest/dickies/.sh_history
Sent
Retry # 1  Directory-->            34,816 /adsmtest/dickies/big  Sent
Retry # 1  Normal File-->          35,712 /adsmtest/dickies/big/F10008
Sent
Normal File-->          10,176 /adsmtest/dickies/big/F10009  Sent
Normal File-->          64,608 /adsmtest/dickies/big/F10010  Compressed
Data Grew
Retry # 2  Normal File-->          35,712 /adsmtest/dickies/big/F10008
Sent
Retry # 1  Normal File-->          10,176 /adsmtest/dickies/big/F10009
Sent
Retry # 1  Normal File-->          64,608 /adsmtest/dickies/big/F10010
Sent
Normal File-->          57,488 /adsmtest/dickies/big/F10011  Compressed
Data Grew
Retry # 2  Normal File-->          64,608 /adsmtest/dickies/big/F10010
Sent
Retry # 1  Normal File-->          57,488 /adsmtest/dickies/big/F10011
Sent
Normal File-->          25,888 /adsmtest/dickies/big/F10053  Sent
Normal File-->          55,856 /adsmtest/dickies/big/F10054  Compressed
Data Grew
Retry # 2  Normal File-->          57,488 /adsmtest/dickies/big/F10011
Sent
Retry # 1  Normal File-->          25,888 /adsmtest/dickies/big/F10053
Sent
Retry # 1  Normal File-->          55,856 /adsmtest/dickies/big/F10054
Sent
Normal File-->          28,560 /adsmtest/dickies/big/F10055  Sent
Normal File-->          55,152 /adsmtest/dickies/big/F10167  Compressed
Data Grew
Retry # 2  Normal File-->          55,856 /adsmtest/dickies/big/F10054
Sent
Retry # 1  Normal File-->          28,560 /adsmtest/dickies/big/F10055
Sent
Retry # 1  Normal File-->          55,152 /adsmtest/dickies/big/F10167
Sent

where it would seem that files are being sent repeatedly to the server.



It's almost as if ADSM starts a tranaction to the server with a file, and
then adds on additional files. If the additional file expands during
compression, then the entire transaction is restarted.



This owuld almost seem to imply that one probably wants to disable
compression unless you know that the majorityt of data being backed IS
compressible, especially if the idea is to use compression to reduce the
LAN traffic.



Comments??



Peter Thomas