On Thu, Mar 16, 2006 at 10:43:30AM +0100, Thomas Widhalm wrote:
> On Thu, 2006-03-16 at 10:32 +0100, Paul Bijnens wrote:
> > On 2006-03-16 09:12, Thomas Widhalm wrote:
> > > 
> > > [root@amanda root]# amadmin IS balance
> > > 
> > >  due-date  #fs    orig KB     out KB   balance
> > > ----------------------------------------------
> > >  3/16 Thu    0          0          0      ---
> > >  3/17 Fri    0          0          0      ---
> > >  3/18 Sat    0          0          0      ---
> > >  3/19 Sun    1    2960985    2960984     -8.7%
> > >  3/20 Mon    2   18406150   14296059   +341.0%
> > >  3/21 Tue    6    4925020    2194810    -32.3%
> > > ----------------------------------------------
> > > TOTAL        9   26292155   19451853   3241975
> > >   (estimated 6 runs per dumpcycle)
> > 
> > So from the above, amanda does not need to dump anything at
> > level 0 for the next three days, but she looks ahead and sees
> > that on 3/20 she expects to dump 14 Gbyte.  So during the
> > next run she will promote some filesystem to that run, in the
> > hope that to lower the amount of work on 3/20.  She tries to
> > dump 19 GByte / 6 each time.
> > 
> > But I guess that one of those filesystems due on 3/20 is a
> > very large one.  And so Amanda is always out of balance.
> > She tries to dump 1/6th each day, promoting everything,
> > except the large one. But the work is for nothing because that
> > one filesystem is just way bigger than 1/6 of the work.
> > 
> > Does that make sense in your config?
> 
> Oh yes, it really does. In every config there is at least one filesystem
> much bigger than all others. If this is a problem for amanda, I
> understand my issues.
> 
> > > 
> > > [root@amanda root]# amoverview IS
> > >          date                 03 03 03 03 03 03 03 03 03
> > > host     disk                 08 09 10 11 12 13 14 15 16
> > > 
> > > amanda.e /                     0  0  0  0  0  0  1  0  1
> > > amanda.e /boot                    0  0  0  0  0  1  0  1
> > > commodo. /                     1  2  2  2  2  2  2  1  1
> > > linglab. /                        1  1  1  1  1  0  1  1
> > > linglab. /home                    1  1  1  1  1  1  1  1
> > > psyserv0 /                        0  0  0  0  0  1  0  1
> > > psyserv0 /boot                    0  0  0  0  0  1  0  1
> > > psyserv0 /usr                     0  0  0  0  0  1  0  1
> > > psyserv0 /var                     0  0  0  0  0  1  0  1
> > 
> > How large is the level 0 dump of linglab. root filesystem?
> > Is that the enormous chunk that cannot Amanda has difficulty
> > to balance?
> 
> This seems correct to me. linglab is rather big.
> 
> > > Please remember that IS is just one of, for now, 4 configs which are run
> > > daily.
> > 
> > Are those 4 configs on 4 servers?
> >   Any reason not to consolidate them
> 
> 4 configs on one server, backing up different hosts each.
> 

Just a clarification on why Paul suggests consolidating the
configs if you can - a recommendation I strongly endorse.

In the IS config about there are 20GB total compressed data
that must be backed to ensure level 0 backups during the 6 run
dumpcycle.  For balance amanda would like to do this at about
3GB/run.  Your two DLE from linglab you say are huge, lets
pretend about 6 or 7GB each.  Then the other 7 DLE would
average about 1GB.

Note the two linglab DLE's did not get frequent level 0's.
Further, on the separate days days they did get a level 0,
nothing else did.  On the other 4 runs of the dumpcycle
amanda still tries to get to the 3GB balance and thus
repeatedly promotes smaller DLE's to push the total up.

Lets assume each of your 4 configs is similar in makeup and
data distribution to IS.  By consolidating the 4 you would
give amanda 36 DLEs to work with, 80GB total to backup,
14GB/run to get level 0's, 8 DLEs with large sizes of
about 1/2 the daily balance total each and 28 small DLEs
to fill in the gaps.

Now amanda can schedule one or two of the big DLEs for
level 0's each run and it can fill in the low days with
level 0's of the 28 small DLEs.  I think you will find
amanda would then give you a good daily balance and
stop the frequent promotion no longer needed to achieve
a pseudo-balance.

Should you decide to consolidate, I'd recommend you NOT
do it all in one shot.  Each new addition would be a "new
disk" to amanda and would have to immediately get a level 0.
You probably don't want all 36 DLEs getting level 0s at
the same time.

And don't forget that inparallel parameter.

-- 
Jon H. LaBadie                  jon AT jgcomp DOT com
 JG Computing
 4455 Province Line Road        (609) 252-0159
 Princeton, NJ  08540-4322      (609) 683-7220 (fax)