Amanda-Users

Re: filesystem limit?

2004-03-16 18:15:57
Subject: Re: filesystem limit?
From: Gene Heskett <gene.heskett AT verizon DOT net>
To: amanda-users AT amanda DOT org
Date: Tue, 16 Mar 2004 18:09:52 -0500
On Tuesday 16 March 2004 16:02, Geoff Swavley wrote:
>Hi All,
>
>I was wondering if anyone could tell me why amanda seems to split
>my filesystem into 2 "holding" files? An error has occurred so these
> filesare left on the disk. Here is the filesystem listing:
>at the files:
>
>46592    1 drwxr-xr-x   4 amanda   sys           512 Mar 17 00:45
>/holding2/schedule8
>209664    1 drwx------   2 amanda   sys           512 Mar 13 05:22
>/holding2/schedule8/20040312
>209666 2301584 -rw-------   1 amanda   sys      2355658752 Mar 13
> 05:22 /holding2/schedule8/20040312/pokolbin._u20.0.1
>209665 31472648 -rw-------   1 amanda   sys      32212254720 Mar 13
> 05:07 /holding2/schedule8/20040312/pokolbin._u20.0
>203840    1 drwx------   2 amanda   sys           512 Mar 17 00:45
>/holding2/schedule8/20040317
>
>Has it got something to do with  a 32GB limit? What does the ".1"
> and ".0" refer to?
>Obviously the ".1" ias the 2nd file of the filesystem??
>
>thanks all. Here is the error email I received:
>
>Subject:     schedule8 AMANDA MAIL REPORT FOR March 12, 2004
>   Date:        Sat, 13 Mar 2004 08:26:54 +1100 (EST)
>   From:       Amanda Archiving Server <amanda AT dipnr.nsw.gov DOT au>
>     To:        aman AT dipnr.nsw.gov DOT au, amanda AT dipnr.nsw.gov DOT au
>
>
>These dumps were to tape schedule8-WEEK3.
>*** A TAPE ERROR OCCURRED: [[writing file: short write]].
>Some dumps may have been left in the holding disk.
>Run amflush to flush them to tape.
>The next tape Amanda expects to use is: schedule8-WEEK4.
>
>FAILURE AND STRANGE DUMP SUMMARY:
>  pokolbin   /u20 lev 0 FAILED [out of tape]
>
>
>STATISTICS:
>                          Total       Full      Daily
>                        --------   --------   --------
>Estimate Time (hrs:min)    0:00
>Run Time (hrs:min)         9:41
>Dump Time (hrs:min)        6:34       6:34       0:00
>Output Size (meg)       32966.5    32966.5        0.0
>Original Size (meg)     32966.5    32966.5        0.0
>Avg Compressed Size (%)     --         --         --
>Filesystems Dumped            1          1          0
>Avg Dump Rate (k/s)      1426.7     1426.7        --
>
>Tape Time (hrs:min)        0:00       0:00       0:00
>Tape Size (meg)             0.0        0.0        0.0
>Tape Used (%)               0.0        0.0        0.0
>Filesystems Taped             0          0          0
>Avg Tp Write Rate (k/s)     --         --         --
>
>USAGE BY TAPE:
>  Label                 Time      Size      %    Nb
>  schedule8-WEEK3       0:00       0.0    0.0     0
>
>
>NOTES:
>  taper: tape schedule8-WEEK3 kb 31851520 fm 1 writing file: short
> write driver: going into degraded mode because of tape error.
>
>
>DUMP SUMMARY:
>                                     DUMPER STATS            TAPER
> STATS HOSTNAME     DISK        L ORIG-KB OUT-KB COMP% MMM:SS  KB/s
> MMM:SS  KB/s --------------------------
> --------------------------------- ------------ pokolbin     /u20   
>     0 3375766433757664   --  394:211426.7  FAILED  ----
>
>(brought to you by Amanda version 2.4.4p2)

This looks to me as if you ran out of tape.  Did you just add these to 
the disklist?  How did you determine the tapetype?  Is the tape 
drives hardware compressor enabled?  Amtapetype can determine this 
and tell you.

The latter case in particular can and will hide the true capacity of a 
tape from amanda.  Please understand that amanda counts bytes sent to 
the drive when making its judgements as to tape fitting.  Enabling 
the drives compressor makes amanda think the tape is much larger than 
it really is so this users recommendation is and always has been to 
shut off any hardware compression and use only software compression.

Yes, it takes time and cpu horsepower to do software compression, but 
it can beat the socks off the hardware compressor for most 
compressable data, sometimes smunching things down to <10% of the 
original byte count.  Since amanda tracks the size of the compressed 
file, amanda then knows exactly how much has been moved to the media 
and very very rarely will amanda hit an EOT then.

When adding a new path to amanda's disklist, I always set the 
compression on, then look at the mailed report from the next run.  If 
compression only saved 5 or 10 percent, then that data in that path 
should be considered as already compressed, like a directory full of 
rpm's or tar.gz's.  Binary executables generally will fall into this 
category too, so one shouldn't waste the cpu horsepower to even try.  
OTOH, your /etc dir will probably smunch to <20% of its original size 
and should be compressed on most systems.

Also, keep in mind that amanda, generally speaking, cannot handle a 
disklist entry thats truely bigger than the tape because amanda 
cannot, without patching, span a single entry across more than one 
tape.  Because that involves tape operations that amanda may not have 
full control over, thereby endangering the data, that limitation 
isn't something the authors are working to fix as its not really 
considered broken by anyone except the fellow who bought a too small 
tape drive.

Disklist entries can be broken down into smaller pieces if you are 
using tar, and in your case it appears that would be the 
recommendation.  For instance, I treat each subdir in my 46Gb /usr as 
a seperate entry, and my 4 tape changer, equipt with a 4Gb DDS2 drive 
can then handle nearly 70Gb of data with an 8 day dumpcycle.  
Normally using only 1 tape a nite, tonight is going to be interesting 
because amanda has been shut down for the last 6 weeks as I've been 
out of pocket & didn't want to saddle the missus with the tape 
changing while I was gone.  So tonight it gets to use 4 tapes, but 
will only find 2 unless I reload the magazine, so I'll have to flush 
in the morning.  This will continue till amanda has played catchup.  
Minor detail, generally speaking.

[...]

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
99.22% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com attornies please note, additions to this message
by Gene Heskett are:
Copyright 2004 by Maurice Eugene Heskett, all rights reserved.

<Prev in Thread] Current Thread [Next in Thread>