Amanda-Users

Re: more problems with dumper/taper/flush retries failures

2005-11-09 07:59:23
Subject: Re: more problems with dumper/taper/flush retries failures
From: Jean-Francois Malouin <Jean-Francois.Malouin AT bic.mni.mcgill DOT ca>
To: Joshua Baker-LePain <jlb17 AT duke DOT edu>
Date: Wed, 9 Nov 2005 07:47:56 -0500
Hello Joshua,

* Joshua Baker-LePain <jlb17 AT duke DOT edu> [20051109 07:07]:
> On Wed, 9 Nov 2005 at 6:26am, Jean-Francois Malouin wrote
> 
> >< I'm having a LOT of problems right now with failed retries and to me
> >it makes no sense that amanda should not be able to dump/tape or flush
> >an image of a DLE weather it's in the holding disk or still sitting
> >remotely on a client, knowing that client-server communication is
> >established, the DLE can be stuffed in a tape, data timeouts are
> 
> Yours can't be.  See below.
> 
> >within limits, etc. My understanding is that amanda should try very
> >hard to tape a DLE but right now it drops the puck fairly easily
> >without much fighting. My take: if a client DLE hits EOT amanda should
> >at least retry once more and not simply go on to the next DLE.
> 
> That's exactly what it's doing.  It tried twice, then gives up.

I meant to say that the flush is the result of the regular amanda runs
not completing normally for this DLE. I bumped the holdding disk usage
to 100GB so that the dump image would make it to the holdding first.
At least I have the full in there if something bad happens.

> 
> >OK, sorry, problem follows, excuse my venting.../>
> >
> >I have this client DLE that refuses to be taped. It's in the holdding
> >this however (~80GB) and trying to flush the failed dumped I get the
> >email output from amanda:
> *snip*
> 
> >NOTES:
> > taper: tape stk_40-conf7-000025 kb 81303232 fm 1 writing file: No space 
> > left on device
> > taper: retrying yorick:/data/mril/mril5/bojana.0 on new tape: [writing 
> > file: No space left on device]
> > taper: tape stk_40-conf7-000026 kb 81267648 fm 1 writing file: No space 
> > left on device
> > taper: retrying yorick:/data/mril/mril5/bojana.0 on new tape: [writing 
> > file: No space left on device]
> > taper: tape stk_40-conf7-000017 kb 0 fm 0 [OK]
> 
> So, it tried writing /data/mril/mril5/bojana to stk_40-conf7-000025, but 
> it hit EOT after writing 81303232 KiB.  So, it cycled to the next tape 
> (stk_40-conf7-000026) and tried again.  This time it hit EOT after writing 
> 81267648 KiB.  At that point it gave up on /data/mril/mril5/bojana.  It 
> then cycled to the next tape (not really necessary), but, finding no other 
> images to try to write, finished.  Simply put, that image is too big to 
> fit on your tapes.

That should not be the case. It's below LTO1 tape capacity for sure.
This is what has me puzzled. I will investigate this further when I
show up at work later today.

> 
> If you think that shouldn't be the case, you need to find out why it's 
> happening.  Do you have both software and hardware compression enabled?

I have a 8 amanda configurations in parallel using 8 LTO1 Ultrium tape
drives (100GB) and I've never encountered this problem before, that
is, until I upgraded to 2.4.5 (essentially for the new estimate
capabilities). I have around 14TB being backed up using this scheme
and I've never had so many problems as of lately. 

> This isn't amanda's fault -- it's doing the best it can to get that image 
> on tape.  It just won't fit.

I will rerun amtapetype on that particular drive later today to see
what's going on. Maybe this tape drive is flaky or needs cleaning but
I pretty sure I did clean it a little while ago...

In the mean time I've reconfigured amanda to use another drive and
I've just started another flush.

Thank you for your help and input Joshua.
I've been staring at this for a long time and I know I might have lost
the tree in the forest (or is it the way around? Time for more coffee!
:)

jf

> 
> -- 
> Joshua Baker-LePain
> Department of Biomedical Engineering
> Duke University

--
Jean-François Malouin, Email: <Jean-Francois.Malouin AT bic.mni.mcgill DOT ca>
McConnell Brain Imaging Centre      Voice:               (514) 398-8924
Montréal Neurological Institute     Fax:                 (514) 398-8948
Montréal, Québec, H3A 2B4, Canada   http://www.bic.mni.mcgill.ca/~malin

<Prev in Thread] Current Thread [Next in Thread>