Hi Everyone,
Today I run another 'Full' set and had opportunity to test what Stefan
said. I still saw single dumping process per host but something got
changes and I found two dumping processes on the same computer in
parallel. I'll run amplot on current Full and publish amfiles later
today.

Thanks, Alexc

This is when two dumpers were running on the same host
---------------------
[amanda@backup Full3]$ amstatus Full3 --dumping
Using /var/lib/amanda/Full3/amdump from Tue Jun 21 11:05:05 IDT 2005
comp1:/dev/sd3a      0  1375797k dumping    71808k (  5.22%) (12:18:30)
comp1:/dev/sd5a      0   208657k dumping   184320k ( 88.34%) (12:18:15)

This is usual picture
---------------------
[amanda@backup Full3]$ amstatus Full3 
Using /var/lib/amanda/Full3/amdump from Tue Jun 21 11:05:05 IDT 2005

comp1:/dev/sd0a      0  1438102k wait for dumping 
comp1:/dev/sd0h      0  1298874k wait for dumping 
comp1:/dev/sd1f      0  1536235k wait for dumping 
comp1:/dev/sd1h      0  1616521k wait for dumping 
comp1:/dev/sd2a      0    16350k finished (11:37:05)
comp1:/dev/sd2f      0  1320298k wait for dumping 
comp1:/dev/sd2g      0   156340k finished (11:46:46)
comp1:/dev/sd2h      0      180k finished (11:39:15)
comp1:/dev/sd3a      0  1375797k dumping   169216k ( 12.30%) (12:18:30)
comp1:/dev/sd3h      0  1661872k wait for dumping 
comp1:/dev/sd4a      0   172410k finished (12:13:52)
comp1:/dev/sd4g      0  1490640k wait for dumping 
comp1:/dev/sd4h      0   929427k wait for dumping 
comp1:/dev/sd5a      0   208650k finished (12:31:05)
comp1:/dev/sd5g      0  1192205k wait for dumping 
comp1:/dev/sd5h      0  1706654k wait for dumping 
comp4:/dev/rd/c0d0p1 0   259290k finished (11:52:16)
comp4:/dev/rd/c0d0p3 0 15348286k wait for dumping 
comp4:/dev/rd/c0d0p5 0  2507290k wait for dumping 
comp4:/dev/rd/c0d0p6 0    67240k finished (11:47:56)
comp4:/dev/rd/c0d0p7 0  1826090k finished (12:29:55)
comp2:sda1           0   168060k finished (11:48:51)
comp2:sda5           0   706340k finished (11:56:05)
comp2:sda6           0   857000k finished (12:09:49)
comp2:sda7           0      440k finished (11:32:59)
comp2:sda8           0   107690k finished (11:47:32)
comp3:sda1           0   167710k finished (11:49:47)
comp3:sda5           0   656550k finished (11:59:37)
comp3:sda6           0   566950k finished (12:02:50)
comp3:sda7           0      450k finished (11:33:01)
comp3:sda8           0   196280k finished (11:50:52)
localhost:sda1       0   440160k finished (12:05:13)
localhost:sda5       0       50k finished (11:33:03)
localhost:sda6       0   574320k finished (12:12:55)
localhost:sda7       0  1134510k finished (12:20:08)

SUMMARY          part      real  estimated
                           size       size
partition       :  35
estimated       :  35             41569897k
flush           :   0         0k
failed          :   0                    0k           (  0.00%)
wait for dumping:  12             32046404k           ( 77.09%)
dumping to tape :   0                    0k           (  0.00%)
dumping         :   1    169216k   1375797k ( 12.30%) (  0.41%)
dumped          :  22   8283060k   8147696k (101.66%) ( 19.93%)
wait for writing:   0         0k         0k (  0.00%) (  0.00%)
wait to flush   :   0         0k         0k (100.00%) (  0.00%)
writing to tape :   0         0k         0k (  0.00%) (  0.00%)
failed to tape  :   0         0k         0k (  0.00%) (  0.00%)
taped           :  22   8283060k   8147696k (101.66%) ( 19.93%)
  tape 1        :  22   8283060k   8147696k ( 14.58%) Full308
15 dumpers idle : no-bandwidth
taper idle
network free kps:      1776
holding space   :  70809996k ( 98.09%)
 dumper0 busy   :  0:56:29  ( 96.83%)
 dumper1 busy   :  0:12:36  ( 21.62%)
 dumper2 busy   :  0:06:27  ( 11.07%)
 dumper3 busy   :  0:00:00  (  0.01%)
   taper busy   :  0:45:41  ( 78.32%)
 0 dumpers busy :  0:00:00  (  0.00%)
 1 dumper busy  :  0:41:06  ( 70.46%)        no-bandwidth:  0:41:06
(100.00%)
 2 dumpers busy :  0:17:13  ( 29.53%)        no-bandwidth:  0:17:13
(100.00%)
 3 dumpers busy :  0:00:00  (  0.00%)
 4 dumpers busy :  0:00:00  (  0.01%)

diskspace:
---------------------
[amanda@backup Full3]$ df -h | grep amanda
/dev/hda1             111G   39G   70G  36% /amanda

disklist:
---------------------
comp1 /dev/sd2a act-full
comp1 /dev/sd2g act-full
comp1 /dev/sd2f act-full
comp1 /dev/sd2h act-full
comp1 /dev/sd1f act-full
comp1 /dev/sd1h act-full
comp1 /dev/sd3a act-full
comp1 /dev/sd3h act-full
comp1 /dev/sd0a act-full
comp1 /dev/sd0h act-full
comp1 /dev/sd4a act-full
comp1 /dev/sd4g act-full
comp1 /dev/sd4h act-full
comp1 /dev/sd5a act-full
comp1 /dev/sd5g act-full
comp1 /dev/sd5h act-full

localhost sda1 act-full
localhost sda5 act-full
localhost sda6 act-full
localhost sda7 act-full

comp2 sda1 act-full
comp2 sda8 act-full
comp2 sda7 act-full
comp2 sda5 act-full
comp2 sda6 act-full

comp3 sda1 act-full
comp3 sda8 act-full
comp3 sda7 act-full
comp3 sda5 act-full
comp3 sda6 act-full

comp4 /dev/rd/c0d0p1 act-full
comp4 /dev/rd/c0d0p7 act-full
comp4 /dev/rd/c0d0p6 act-full
comp4 /dev/rd/c0d0p5 act-full
comp4 /dev/rd/c0d0p3 act-full


amanda.conf:
---------------------
org "Full3"
mailto "amanda"
dumpuser "amanda"
inparallel 16            <----- increased to 16
netusage  600 Kbps
dumpcycle 0 weeks
runspercycle 4 weeks
tapecycle 1000 tapes
bumpsize 20 Mb
bumpdays 1
bumpmult 4
etimeout 1200
runtapes 1
tapedev "/dev/nst0"
tapetype dat72comp      # what kind of tape it is (see tapetypes below)
labelstr "^Full3[0-9][0-9]$"    # label constraint regex: all tapes must match

holdingdisk hd1 {
    comment "main holding disk"
    directory "/amanda/Full3"   # where the holding disk is
    use -1 Gb            <----- reduced to 1Gb, Paul Bijnens recommended that. 
}
reserve 5
infofile "/var/lib/amanda/Full3/curinfo"        # database filename
logdir   "/var/lib/amanda/Full3"                # log directory
indexdir "/var/lib/amanda/Full3/index"  # index directory

define tapetype dat72 {
    comment "HP DAT 72"
    length 35480 mbytes
    filemark 0 kbytes
    speed 2906 kps
}

define tapetype dat72comp {
    comment "HP DAT 72 with compression"
    length 55480 mbytes
    filemark 0 kbytes
    speed 2906 kps
}

define tapetype dat72-nocomp {
    comment "HP DAT 72 (hardware compression off)"
    length 35429 mbytes
    filemark 0 kbytes
    speed 3027 kps
}

define dumptype global {
    comment "Global definitions"
    index yes
    record no
    maxdumps 5          <----- decreased to 5
}

define dumptype act-full {
    global
    comment "Full dump of this filesystem always"
    compress none
    priority high
    dumpcycle 0
}

define dumptype act-tar {
    global
    program "GNUTAR"
    compress none
    exclude list "/etc/amanda/exclude.gtar"
    priority high
}


define dumptype always-full {
    global
    comment "Full dump of this filesystem always"
    compress none
    priority high
    dumpcycle 0
}

define dumptype root-tar {
    global
    program "GNUTAR"
    comment "root partitions dumped with tar"
    compress none
    index
    exclude list "/usr/local/lib/amanda/exclude.gtar"
    priority low
}

define dumptype user-tar {
    root-tar
    comment "user partitions dumped with tar"
    priority medium
}

define dumptype high-tar {
    root-tar
    comment "partitions dumped with tar"
    priority high
}

define dumptype comp-root-tar {
    root-tar
    comment "Root partitions with compression"
    compress client fast
}

define dumptype comp-user-tar {
    user-tar
    compress client fast
}

define dumptype holding-disk {
    global
    comment "The master-host holding disk itself"
    holdingdisk no # do not use the holding disk
    priority medium
}

define dumptype comp-user {
    global
    comment "Non-root partitions on reasonably fast machines"
    compress client fast
    priority medium
}

define dumptype nocomp-user {
    comp-user
    comment "Non-root partitions on slow machines"
    compress none
}

define dumptype comp-root {
    global
    comment "Root partitions with compression"
    compress client fast
    priority low
}

define dumptype nocomp-root {
    comp-root
    comment "Root partitions without compression"
    compress none
}

define dumptype comp-high {
    global
    comment "very important partitions on fast machines"
    compress client best
    priority high
}

define dumptype nocomp-high {
    comp-high
    comment "very important partitions on slow machines"
    compress none
}

define dumptype nocomp-test {
    global
    comment "test dump without compression, no /etc/dumpdates recording"
    compress none
    record no
    priority medium
}

define dumptype comp-test {
    nocomp-test
    comment "test dump with compression, no /etc/dumpdates recording"
    compress client fast
}

define interface local {
    comment "a local disk"
    use 1000 kbps
}

define interface eth0 {
    comment "10 Mbps ethernet"
    use 400 kbps
}




On Mon, 2005-06-20 at 12:37 +0200, Stefan G. Weichinger wrote: 
> Vects wrote:
> 
> > I changed inparallel and maxdumps but it didn't help.
> > See below the configuration files of amanda.
> > 
> > inparallel 10
> 
> > define dumptype global {
> >     comment "Global definitions"
> >     index yes
> >     record no
> >     maxdumps 10
> > }
> > 
> > define dumptype act-full {
> >     global
> >     comment "Full dump of this filesystem always"
> >     compress none
> >     priority high
> >     dumpcycle 0
> > }
> 
> I would start testing without any spindle-numbers inside disklist.
> Until you get parallelity into your backups there is no need to restrict
> it ;-)
> 
> Also re-read the manpage for the description of the two parameters:
> 
> "maxdumps" limits the number of dumper processes per DLE, while
> "inparallel" limits the overall number of dumper processes running in
> parallel.
> 
> It won't hurt to increase "inparallel" to 16 or something, IMHO it
> should be set higher than "maxdumps" for a given dumptype, otherwise it
> could happen, that one greedy DLE grabs all "inparallel"
> dumper-processes and your parallelity is gone (unlikely, I know).
> 
> So:
> 
> - remove spindle-numbers for a start
> - set inparallel to 16
> - set maxdumps to 5 inside your global-dumptype
> - watch the behavior by using amstatus, and look out for things like
> "client-restrained" which tell you more ...
> 
> You might also use amplot to generate graphs for analysis of your
> bottlenecks, which analyses your amdump-file (which you might also
> simply show us, as Paul just now suggested).
> 
> Stefan G. Weichinger
> sgw AT amanda DOT org
> 
> 
> 
>