Amanda-Users

Re: Amanda compression questions

2007-06-01 12:38:43
Subject: Re: Amanda compression questions
From: Chris Hoogendyk <hoogendyk AT bio.umass DOT edu>
To: "McGraw, Robert P." <rmcgraw AT purdue DOT edu>
Date: Fri, 01 Jun 2007 12:36:15 -0400

McGraw, Robert P. wrote:
>
> My configuration:
>
> zorn->[8] > amadmin daily version
>
> build: VERSION="Amanda-2.5.2-20070530"
>
> BUILT_DATE="Thu May 31 09:14:26 EDT 2007"
>
> BUILT_MACH="SunOS zorn.math.purdue.edu 5.10 Generic_118833-03 sun4u
> sparc SUNW,Sun-Fire-280R"
>
> CC="/pkgs/gcc-3.4.3/bin/gcc"
>
> CONFIGURE_COMMAND="'./configure' 'CC=/pkgs/gcc-3.4.3/bin/gcc'
> 'MT=/opt/csw/bin/mt' 'MTF=-f' 'MTX=/opt/csw/sbin/mtx'
> 'PERL=/opt/csw/bin/perl'
> '--prefix=/local/Amanda/amanda-2.5.2-20070530' '--with-user=amanda'
> '--with-group=operator' '--with-gnutar=/opt/csw/bin/gtar'
> '--with-gnutar-listdir=/var/amanda/gnutar-lists'
> '--with-includes=/opt/csw/include' '--with-libraries=/opt/csw/lib'
> '--with-index-server=zorn'
> '--with-smbclient=/pkgs/samba/bin/smbclient'
> '--with-tape-device=/dev/rmt/1bn'
> '--with-changer-device=/dev/changer/1' '--with-maxtapeblocksize=2048'"
>
> My tape unit is a Sun C4 38 slot and 1 LTO2 tape drive. My tapetype is
> defined as followed
>
> define tapetype LTO2-HWC {
>
> comment "LTO-2-Hardware Compression on."
>
> blocksize 1024 kbytes
>
> length 300000 mbytes #200G
>
> filemark 0 kbytes
>
> speed 27315 kps #27 Mb/s
>
> }
>
> “length 300000 mbytes” was calculated by 1.5 times the actual size of
> 200GB.
>
> Up to now my dumptype has had compress equal to none. I decided to let
> the hardware do the compression work. The problem with this is the
> calculation of the length is just a guess and seems to cause problems
> when trying to fit files on the tape. In some cases amanda thinks it
> has more tape than it really does so it tries to write a bigger file
> to the last part of the tape and fails.
>
> I am thinking of changing my setup to do software compression and see
> if this will give me better tape usage.
>
> Q1) I have asked Sun and the makers of the LTO2 drive and it seems
> there is no way to turn hardware compression on or off on this unit,
> it is always on. From my reading in the Amanda forum it seems that the
> LTO2 drive will sense if the data is compressed and turn off hardware
> compression if the data is already compressed. Is this correct?
>
> Q2) Any suggestions, pros or cons in going this direction? I guess the
> trade off is that I would get better tape usage at the expense of
> compute time to do the software compression.
>
> Q3) From the config.log it seems the default program for compression
> is gzip and it uses –fast or –best. From gzip man page it says I can
> use -#x where x is between 1 (fast) 9 (best). Is there a way to set
> the compress number without having to use the “compress server custom”?
>

I'll let someone else comment on Q1 and Q3.

I've been using amanda 2.5.1p3 on a Sun E250 with Solaris 9 and a Sony
AIT5 16 slot changer. I set it up with server side software compression.
The AIT5 has a native capacity of 400G without compression, and that is
how I defined it in the tapetype. Since my E250 is dedicated to backup,
has dual processors, and has plenty of disk cache (two 300G lvd scsi
drives), I wasn't concerned about issues with having the server doing
compression. The result is almost perfect planning on the part of
amanda. After some experience with my DLEs, it has a very good idea how
things are going to turn out; and, after doing its own compression, it
knows exactly how much it is trying to put on a tape.

I think the trade-offs are way in favor of software compression. You
just never know how compressible something is going to be. Partitions
loaded with images that are in an already compressed format won't
compress. Web partitions with no images will compress beautifully.
Without some magical tape interface that gives you back how much
something compressed and how much tape is left, hardware compression
only leaves you the option of "write until you hit eof", which makes it
impossible to decide whether you can fit a particular parition or not.

I'm presuming your 280R is dual processor and that you have given it
significant disk caching space.



---------------

Chris Hoogendyk

-
   O__  ---- Systems Administrator
  c/ /'_ --- Biology & Geology Departments
 (*) \(*) -- 140 Morrill Science Center
~~~~~~~~~~ - University of Massachusetts, Amherst 

<hoogendyk AT bio.umass DOT edu>

--------------- 

Erdös 4



<Prev in Thread] Current Thread [Next in Thread>