Amanda-Users

Re: taperalgo "largestfit" appears to be backwards.

2004-10-30 06:47:54
Subject: Re: taperalgo "largestfit" appears to be backwards.
From: Gene Heskett <gene.heskett AT verizon DOT net>
To: Joe Rhett <jrhett AT meer DOT net>
Date: Sat, 30 Oct 2004 06:40:25 -0400
On Friday 29 October 2004 23:31, Joe Rhett wrote:
>I just changed the taperalgo parameter in the amanda configuration
> per someone's suggestion (regarding a different problem) and have
> noticed that
>
> taperalgo  largestfit
>
>Appears to work backwards.  It is backing up all the smallest
> filesystems first, and putting the biggest ones at the end.  Which
> means that it hits tape end on these, and uses nearly 3x as many
> tapes as before.
>
>Here's an example partial log entry...
>
>FINISH planner date 20041029
>STATS driver startup time 6104.079
>SUCCESS dumper web013.svk.isite.net / 20041029 1 [sec 1.457 kb 64
> kps 43.9 orig-kb 63] SUCCESS dumper web031.svk.isite.net / 20041029
> 1 [sec 1.512 kb 32 kps 21.2 orig-kb 31] SUCCESS dumper
> web012.svk.isite.net / 20041029 1 [sec 1.207 kb 32 kps 26.5 orig-kb
> 31] SUCCESS dumper web014.svk.isite.net / 20041029 1 [sec 1.948 kb
> 32 kps 16.4 orig-kb 31] SUCCESS taper web013.svk.isite.net /
> 20041029 1 [sec 2.366 kb 64 kps 27.0 {wr: writers 3 rdwait 0.000
> wrwait 0.007 filemark 2.357}] SUCCESS dumper anubis.svk.isite.net /
> 20041029 1 [sec 3.288 kb 64 kps 19.5 orig-kb 63] SUCCESS taper
> web031.svk.isite.net / 20041029 1 [sec 2.354 kb 32 kps 13.6 {wr:
> writers 2 rdwait 0.000 wrwait 0.005 filemark 2.348}] SUCCESS taper
> anubis.svk.isite.net / 20041029 1 [sec 2.361 kb 64 kps 27.1 {wr:
> writers 3 rdwait 0.000 wrwait 0.007 filemark 2.352}] SUCCESS dumper
> gamera.svk.isite.net /opt 20041029 1 [sec 5.488 kb 64 kps 11.7
> orig-kb 63] SUCCESS dumper smaug.svk.isite.net / 20041029 1 [sec
> 5.045 kb 64 kps 12.7 orig-kb 63] SUCCESS taper web012.svk.isite.net
> / 20041029 1 [sec 2.354 kb 32 kps 13.6 {wr: writers 2 rdwait 0.000
> wrwait 0.005 filemark 2.348}] SUCCESS taper gamera.svk.isite.net
> /opt 20041029 1 [sec 2.356 kb 64 kps 27.2 {wr: writers 3 rdwait
> 0.000 wrwait 0.007 filemark 2.347}] SUCCESS dumper
> web013.svk.isite.net /opt 20041029 1 [sec 1.717 kb 32 kps 18.6
> orig-kb 31] SUCCESS taper smaug.svk.isite.net / 20041029 1 [sec
> 2.355 kb 64 kps 27.2 {wr: writers 3 rdwait 0.000 wrwait 0.007
> filemark 2.347}] SUCCESS dumper web012.svk.isite.net /usr 20041029
> 1 [sec 2.776 kb 64 kps 23.0 orig-kb 63] SUCCESS taper
> web014.svk.isite.net / 20041029 1 [sec 2.354 kb 32 kps 13.6 {wr:
> writers 2 rdwait 0.000 wrwait 0.005 filemark 2.348}] SUCCESS taper
> web012.svk.isite.net /usr 20041029 1 [sec 2.350 kb 64 kps 27.2 {wr:
> writers 3 rdwait 0.000 wrwait 0.007 filemark 2.342}] SUCCESS taper
> web013.svk.isite.net /opt 20041029 1 [sec 2.340 kb 32 kps 13.7 {wr:
> writers 2 rdwait 0.000 wrwait 0.005 filemark 2.334}]
>
>...etc.
>
>Farther down are 15, 20, and 30 gigabyte filesystems, which it
> scheduled at the very last. One per tape, with no small filesystems
> filling up the tapes.

Hmmm, I'm using the latest 2.4.5 snapshot, with the taperalgo set to 
largestfit, and this:
dumporder "STTT" # specify the priority order of each dumper
                        #   s -> smallest size
                        #   S -> biggest size
                        #   t -> smallest time
                        #   T -> biggest time
                        #   b -> smallest bandwidth
                        #   B -> biggest bandwidth
                        # try "BTBTBTBTBTBT" if you are not holding
                        # disk constrained
                        # BUT, if you want streaming, start with the
                        # big ones and work down

And a typical backup session will start with the largest and work down, like 
this:
[root@coyote nist]# ls -l /amandatapes/Dailys/slot16
total 1257120
-rw-------  1 amanda disk        10 Oct 26 08:57 00000-Dailys-16
-rw-------  1 amanda disk     32768 Oct 26 08:57 00000.Dailys-16
-rw-------  1 amanda disk        14 Oct 26 08:57 00001-gene._root.0
-rw-------  1 amanda disk 348618752 Oct 26 08:57 00001.gene._root.0
-rw-------  1 amanda disk        13 Oct 26 08:57 00002-coyote._home.0
-rw-------  1 amanda disk 259850240 Oct 26 08:57 00002.coyote._home.0
-rw-------  1 amanda disk        13 Oct 26 08:57 00003-coyote._opt.0
-rw-------  1 amanda disk 225804288 Oct 26 08:57 00003.coyote._opt.0
-rw-------  1 amanda disk        13 Oct 26 08:57 00004-coyote._usr_include.0
-rw-------  1 amanda disk 116326400 Oct 26 08:57 00004.coyote._usr_include.0
-rw-------  1 amanda disk        13 Oct 26 08:57 00005-coyote._tmp.0
-rw-------  1 amanda disk  35782656 Oct 26 08:57 00005.coyote._tmp.0
-rw-------  1 amanda disk        12 Oct 26 08:57 00006-coyote._usr_libexec.0
-rw-------  1 amanda disk  28278784 Oct 26 08:57 00006.coyote._usr_libexec.0
-rw-------  1 amanda disk        12 Oct 26 08:57 00007-gene._boot.0
-rw-------  1 amanda disk  27590656 Oct 26 08:57 00007.gene._boot.0
-rw-------  1 amanda disk        12 Oct 26 08:58 00008-gene._lib.0
-rw-------  1 amanda disk  22511616 Oct 26 08:58 00008.gene._lib.0
-rw-------  1 amanda disk        12 Oct 26 08:58 00009-coyote._usr_sbin.0
-rw-------  1 amanda disk  19431424 Oct 26 08:58 00009.coyote._usr_sbin.0
-rw-------  1 amanda disk        12 Oct 26 08:58 00010-coyote._sbin.0
-rw-------  1 amanda disk  19005440 Oct 26 08:58 00010.coyote._sbin.0
-rw-------  1 amanda disk        12 Oct 26 08:58 00011-coyote._usr_dlds-rpms.1
-rw-------  1 amanda disk  17629184 Oct 26 08:58 00011.coyote._usr_dlds-rpms.1
-rw-------  1 amanda disk        12 Oct 26 08:58 00012-coyote._dos.0
-rw-------  1 amanda disk  12877824 Oct 26 08:58 00012.coyote._dos.0
-rw-------  1 amanda disk        12 Oct 26 08:58 00013-coyote._var.2
-rw-------  1 amanda disk  10649600 Oct 26 08:58 00013.coyote._var.2
-rw-------  1 amanda disk        12 Oct 26 08:58 00014-coyote._usr_local.2
-rw-------  1 amanda disk  10551296 Oct 26 08:58 00014.coyote._usr_local.2
-rw-------  1 amanda disk        12 Oct 26 08:58 00015-coyote._dev.0
-rw-------  1 amanda disk   9895936 Oct 26 08:58 00015.coyote._dev.0
-rw-------  1 amanda disk        12 Oct 26 08:58 00016-gene._usr_src.1
-rw-------  1 amanda disk   9404416 Oct 26 08:58 00016.gene._usr_src.1
-rw-------  1 amanda disk        12 Oct 26 08:58 00017-coyote._etc.0
-rw-------  1 amanda disk   7929856 Oct 26 08:58 00017.coyote._etc.0
-rw-------  1 amanda disk        12 Oct 26 08:58 00018-gene._bin.0
-rw-------  1 amanda disk   7831552 Oct 26 08:58 00018.gene._bin.0
-rw-------  1 amanda disk        12 Oct 26 08:58 00019-coyote._usr_games.0
-rw-------  1 amanda disk   6455296 Oct 26 08:58 00019.coyote._usr_games.0
-rw-------  1 amanda disk        12 Oct 26 08:58 00020-coyote._bin.0
-rw-------  1 amanda disk   5701632 Oct 26 08:58 00020.coyote._bin.0
-rw-------  1 amanda disk        12 Oct 26 08:58 00021-gene._sbin.0
-rw-------  1 amanda disk   5472256 Oct 26 08:58 00021.gene._sbin.0
-rw-------  1 amanda disk        12 Oct 26 08:58 00022-coyote._usr_src.1
-rw-------  1 amanda disk   4587520 Oct 26 08:58 00022.coyote._usr_src.1
-rw-------  1 amanda disk        12 Oct 26 08:58 00023-coyote._boot.1
-rw-------  1 amanda disk   4292608 Oct 26 08:58 00023.coyote._boot.1
-rw-------  1 amanda disk        12 Oct 26 08:58 00024-gene._home.0
-rw-------  1 amanda disk   4194304 Oct 26 08:58 00024.gene._home.0
-rw-------  1 amanda disk        12 Oct 26 08:58 00025-coyote._lib.1
-rw-------  1 amanda disk   3768320 Oct 26 08:58 00025.coyote._lib.1
-rw-------  1 amanda disk        11 Oct 26 08:58 00026-gene._etc.0
-rw-------  1 amanda disk   2883584 Oct 26 08:58 00026.gene._etc.0
-rw-------  1 amanda disk        11 Oct 26 08:58 00027-gene._usr_local.1
-rw-------  1 amanda disk    884736 Oct 26 08:58 00027.gene._usr_local.1
-rw-------  1 amanda disk        11 Oct 26 08:58 00028-coyote._usr_kerberos.0
-rw-------  1 amanda disk    491520 Oct 26 08:58 00028.coyote._usr_kerberos.0
-rw-------  1 amanda disk        10 Oct 26 08:58 00029-gene._opt.1
-rw-------  1 amanda disk    327680 Oct 26 08:58 00029.gene._opt.1
-rw-------  1 amanda disk        10 Oct 26 08:58 00030-coyote._usr_lib.1
-rw-------  1 amanda disk    294912 Oct 26 08:58 00030.coyote._usr_lib.1
-rw-------  1 amanda disk        10 Oct 26 08:58 00031-coyote._usr_man.0
-rw-------  1 amanda disk    229376 Oct 26 08:58 00031.coyote._usr_man.0
-rw-------  1 amanda disk        10 Oct 26 08:58 00032-coyote._usr_dlds-tgzs.1
-rw-------  1 amanda disk    131072 Oct 26 08:58 00032.coyote._usr_dlds-tgzs.1
-rw-------  1 amanda disk        10 Oct 26 08:58 00033-coyote._usr_music.1
-rw-------  1 amanda disk    131072 Oct 26 08:58 00033.coyote._usr_music.1
-rw-------  1 amanda disk        10 Oct 26 08:58 00034-coyote._usr_X11R6.1
-rw-------  1 amanda disk     98304 Oct 26 08:58 00034.coyote._usr_X11R6.1
-rw-------  1 amanda disk        10 Oct 26 08:58 00035-gene._usr_bin.1
-rw-------  1 amanda disk     98304 Oct 26 08:58 00035.gene._usr_bin.1
-rw-------  1 amanda disk        10 Oct 26 08:58 00036-coyote._usr_pix.1
-rw-------  1 amanda disk     98304 Oct 26 08:58 00036.coyote._usr_pix.1
-rw-------  1 amanda disk        10 Oct 26 08:58 
00037-coyote._usr_i386-glibc21-linux.0
-rw-------  1 amanda disk     98304 Oct 26 08:58 
00037.coyote._usr_i386-glibc21-linux.0
-rw-------  1 amanda disk        10 Oct 26 08:58 00038-coyote._usr_FC2_disk4.1
-rw-------  1 amanda disk     65536 Oct 26 08:58 00038.coyote._usr_FC2_disk4.1
-rw-------  1 amanda disk        10 Oct 26 08:58 00039-coyote._usr_FC2_disk3.1
-rw-------  1 amanda disk     65536 Oct 26 08:58 00039.coyote._usr_FC2_disk3.1
-rw-------  1 amanda disk        10 Oct 26 08:58 00040-coyote._usr_FC2_disk2.1
-rw-------  1 amanda disk     65536 Oct 26 08:58 00040.coyote._usr_FC2_disk2.1
-rw-------  1 amanda disk        10 Oct 26 08:58 00041-coyote._usr_FC2_disk1.1
-rw-------  1 amanda disk     65536 Oct 26 08:58 00041.coyote._usr_FC2_disk1.1
-rw-------  1 amanda disk        10 Oct 26 08:58 00042-coyote._amanda.0
-rw-------  1 amanda disk     65536 Oct 26 08:58 00042.coyote._amanda.0
-rw-------  1 amanda disk        10 Oct 26 08:58 
00043-coyote._misc_fedora_FC3t1.1
-rw-------  1 amanda disk     65536 Oct 26 08:58 
00043.coyote._misc_fedora_FC3t1.1
-rw-------  1 amanda disk        10 Oct 26 08:58 00044-TAPEEND
-rw-------  1 amanda disk     32768 Oct 26 08:58 00044.TAPEEND
-rw-r--r--  1 amanda disk     51200 Sep 30 03:10 configuration.tar
-rw-r--r--  1 amanda disk  54906880 Sep 30 03:10 indices.tar
--------------
And thats a puzzling ls -l of one of my virtual tapes, note
the time stamp on the last 2 items, which should have been
updated to about a minute later than the rest ???

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
99.28% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com attorneys please note, additions to this message
by Gene Heskett are:
Copyright 2004 by Maurice Eugene Heskett, all rights reserved.