Gene,
Stefan,

I should clarify further...

Extracted from "disklist"
samar   /      comp-root
samar   /usr1  comp-user
samar   /usr5/amanda {
        user-tar
        }

Extracted from "amanda.conf"

define dumptype comp-root {
    global
    comment "Root partitions with compression"
    compress client fast
    priority low
}

define dumptype comp-user {
    global
    comment "Non-root partitions on reasonably fast machines"
    compress client fast
    priority medium
}

define dumptype user-tar {
    program "GNUTAR"
    comment "user partitions dumped with tar"
    compress
    index
# exclude list
    priority medium
}

ya know, I'm not seeing it explicitely but the default is to
dump via xfsdump (I have no efs partitions on this box). I
don't believe the dump of either root nor /usr1 is causing a
problem and think that I've got the proper definitions to subdivide
the raid (mounted on /usr5) into directories.

I've written the technical contact for the departement and assigned
/usr5/dumps which is NOT a DLE to act as amanda's holding area. This
however is a far from optimal solution. Raid is nice but still must
suffer from I/O contention and the jukebox/SDLT is on the same bus.

    My initial analysis was correct ? 

    Without a holding area than any DLE being written direct
    to tape will fail and not be retried when the next tape
    is loaded into the drive from the jukebox.

I haven't followed the thread as closely as I could have. There is
a patch for spaning DLE across output/tape volumes ? Would it be
effecatious in a direct to tape senario like this one ? Is it ready
for prime-time ?

                                                thank you,

                                                Brian

On Tue, Nov 30, 2004 at 11:01:19AM -0500, Gene Heskett wrote:
> On Tuesday 30 November 2004 09:43, Brian Cuttler wrote:
> >Stefan,
> >
> >On Tue, Nov 30, 2004 at 12:46:17AM +0100, Stefan G. Weichinger wrote:
> >> Hi, Brian,
> >>
> >> on Montag, 29. November 2004 at 17:51 you wrote to amanda-users:
> >>
> >> BC> I've broken the raid into DLEs based on the top level
> >> directory BC> structure of the (single large) partition. The raid
> >> is .886 PBytes.
> >>
> >> uuuuhh ;-) Sounds tasty.
> >>
> >> BC> Since I don't really have a spool area
> >>
> >> you mean "holding disk" in AMANDA-terms?
> >
> >Yes, "holding disk". In more generic terms a "spooling" area for the
> > dump/tar'd, possibly compressed files before they are DD'd to tape.
> > Or, and this happens periodically, there is some subtletly to the
> > term "spooling" that I've missed or forgetten ? (Which isn't to say
> > the conversation wouldn't be smoother if I'd used to proper jargon
> > in context)
> >
> >> BC> (none officially assigned anyway) the DLE's that saw an EOT
> >> from BC> the SDLT failed, they don't currently span tapes (no
> >> patches in BC> place) and don't retry. Direct to tape doesn't
> >> retry, does it ?
> >>
> >> BC> Is there a way to force "next tape" prior to running the
> >> larger DLEs ?
> >>
> >> I think we could need a bit more details on this.
> >> Based on my assumptions I think you don't have the proper value
> >> for "runtapes" set in your amanda.conf.
> >
> >runtapes is set to 7 - and we in fact consumed multiple tapes during
> > the run.
> >
> >DLE entries...(extract) There are 14 DLEs in all, 12 of the later
> > type.
> >
> >samar   /      comp-root
> 
> Here is the first biter I think.  Tar runs recursively thru the named 
> directories, meaning the above line would also include all below, 
> UNLESS you have added to the comp-root define in your amanda.conf, an 
> "exclude file=/path/to/somefile" that names ./usr1 and ./user5 on 
> seperate lines of that file.
> 
> >samar   /usr1  comp-user
> >samar   /usr5/amanda {
> >        user-tar
> >        }
> >
> >samar   /usr5/liw {
> >        user-tar
> >        }
> >
> >samar   /usr5/hxgao {
> >        user-tar
> >        }
> >
> >Since there was no holding area and we where running direct to tape
> > it seems that we where ok for any DLE that was both started and
> > completed to a single output tape volume.
> 
> Really, the lack of a holding disk area would appear to be a 
> show-stopper in the tape is full scenario.  For data that may not be 
> readily replaceable I think I'd invest in a raid cage big enough to 
> do the job.  The tv station where I still work part time just bought 
> a terrabyte raid thats only about 1" wider and 3" taller than a 
> shoebox, as part of the digitization of the newroom editing 
> functions.  But I don't recall the price.  We also built, back when 
> commodity drives were 160GB, a 320GB 4 drive raid with a hot standby  
> for rsyncing all the business machines in traffic to, and did that 
> for less than $1500.  We could expand that to terrabyte size now for 
> about $600 additional or less, but don't need it just yet.
> 
> >However a DLE that "didn't fit" on the remaining tape didn't restart
> > the TAR when the next tape volume was loaded in the drive nor was
> > it completed in the holding area for a retry of the DD.
> >
> >At least that is what it looks like to me, a not very clearly noted
> > failure.
> >
> >I'll forward the report to you.
> >
> >Unfortunately the raid array is occupied by large data sets (the
> > users are analyzing RNA structures) and there is no knowing in
> > advance how much data an individual will have nor how much of it
> > has been changed between runs.
> >
> >       thank you,
> >
> >       Brian
> >
> >> We would need to know how big your DLEs are as well as what config
> >> you run to be more specific.
> >>
> >> "taperalgo" helps you to get things in order on tapes (and maybe
> >> avoiding and optimizing "smaller DLEs before big ones").
> >>
> >> --
> >> best regards,
> >> Stefan
> >>
> >> Stefan G. Weichinger
> >> mailto:monitor AT oops.co DOT at
> >
> >---
> >   Brian R Cuttler                 brian.cuttler AT wadsworth DOT org
> >   Computer Systems Support        (v) 518 486-1697
> >   Wadsworth Center                (f) 518 473-6384
> >   NYS Department of Health        Help Desk 518 473-0773
> 
> -- 
> Cheers, Gene
> "There are four boxes to be used in defense of liberty:
>  soap, ballot, jury, and ammo. Please use in that order."
> -Ed Howdershelt (Author)
> 99.29% setiathome rank, not too shabby for a WV hillbilly
> Yahoo.com attorneys please note, additions to this message
> by Gene Heskett are:
> Copyright 2004 by Maurice Eugene Heskett, all rights reserved.
---
   Brian R Cuttler                 brian.cuttler AT wadsworth DOT org
   Computer Systems Support        (v) 518 486-1697
   Wadsworth Center                (f) 518 473-6384
   NYS Department of Health        Help Desk 518 473-0773