Amanda-Users

Re: gnutar and sendsize

2005-04-29 08:58:49
Subject: Re: gnutar and sendsize
From: Paul Bijnens <paul.bijnens AT xplanation DOT com>
To: Kee Gohil <kee AT ksl.co DOT uk>
Date: Fri, 29 Apr 2005 14:44:45 +0200
Kee Gohil wrote:
Bear with me on this. I've inherited a system to look after and I'm new to Amanda.

I'm using gnutar to backup a ~50Gb filesystem. Unfortunately it won't backup due to..

/home lev 0 FAILED [dump larger than tape, but cannot incremental dump new disk]

A new disk. OK.


I have compression set on in the dumptype configuration as follows

define dumptype comp-user-tar {
 program "GNUTAR"
 comment "partitions dumped with tar"
 options compress-fast, index, exclude-list "/etc/amanda/exclude.gtar"
 priority medium
}

This seems to be some older syntax. The modern synctax (modern = since
1998 or earlier, I began using amanda in 1998)  would be something like:

define dumptype comp-user-tar {
    program "GNUTAR"
    comment "partions dumped with tar"
    compress fast
    index yes
    exclude-list "/etc/amanda/exclude.gtar"
    priority medium
}


The file system should compress to around 17Gb and will fit on the tape no problem. I had a look at the sendsize report and it showed that during the size estimate, it didn't use compression.

sendsize: argument list: /bin/tar --create --file /dev/null --directory /home --one-file-system --listed-incremental /var/lib/amanda/gnutar-lists/fs0_home_0.new --sparse --ignore-failed-read --totals .

Total bytes written: 49292472320 (46GB, 46MB/s)

I believe that it thinks that the size will be 46Gb which will not fit the tape.

During estimate amanda does not execute the gzip program.
When gnutar notices the "--file /dev/null", it takes some shortcuts
and instead of reading the files themselves, just adds the sizes
for the estimate.  It does read the bytes of the files, and pipes them
through gzip.

Amanda keeps a history of compression ratios for each filesystem
and multiplies that ratio with the above result.
For a new disk amanda has no historical knowledge, and assumes a 50%
compression, give 23 Mbyte in the case above.

But you can tune that value by adding a "comprate" to the dumptype.
Also the above dumptype seems to use some older syntax.
The modern syntax (modern = since 1998 or earlier, I began using amanda
in 1998)  would be something like, adding comprate too:

define dumptype comp-user-tar {
    program "GNUTAR"
    comment "partions dumped with tar"
    compress fast
    comprate 0.40 0.50
    index yes
    exclude-list "/etc/amanda/exclude.gtar"
    priority medium
    comprate 0.40 0.50
}


The above values now assume 0.30*46 = 13.8 Mbyte as the result of
the estimate.

Another way is to split up the filesystem into smaller pieces. See:

http://www.amanda.org/docs/topten.html#id2524456


--
Paul Bijnens, Xplanation                            Tel  +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUM    Fax  +32 16 397.512
http://www.xplanation.com/          email:  Paul.Bijnens AT xplanation DOT com
***********************************************************************
* I think I've got the hang of it now:  exit, ^D, ^C, ^\, ^Z, ^Q, F6, *
* quit,  ZZ, :q, :q!,  M-Z, ^X^C,  logoff, logout, close, bye,  /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt,  abort,  hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e,  kill -1 $$,  shutdown, *
* kill -9 1,  Alt-F4,  Ctrl-Alt-Del,  AltGr-NumLock,  Stop-A,  ...    *
* ...  "Are you sure?"  ...   YES   ...   Phew ...   I'm out          *
***********************************************************************



<Prev in Thread] Current Thread [Next in Thread>