Amanda-Users

Re: two dumps, when combined, larger than the holding disk

2008-04-28 12:20:34
Subject: Re: two dumps, when combined, larger than the holding disk
From: Brian Cuttler <brian AT wadsworth DOT org>
To: Jordan Desroches <jordan.d.desroches AT Dartmouth DOT EDU>
Date: Mon, 28 Apr 2008 11:53:03 -0400
Jordan,

Sounds almost like you have to defeat concurrency, which is
really too bad. I also saw deadlock because of that but never
from a single amdump job with a private partition for holding space.

Probably the easiest way to do that is to use the 4th field
of the disklist file and specify that that two partitions are
on the same spindle, so they will not complete with each other.
The feature is designed to limit head movement, ie do not
concurrently dump partitions on the same spindle, but in this
case you can use it to prevent concurrency because of holding
area limitations.

Or add another holding area ?

On Mon, Apr 28, 2008 at 11:40:08AM -0400, Jordan Desroches wrote:
> Brian,
> 
> Sorry that I was unclear. I have a single amdump task running two DLEs  
> concurrently. The estimate is quite far off due to the variability of  
> compressibility of the data on the DLEs combined with a high turnover  
> rate of data. To complicate matters, I'm running the "fulls only"  
> backup strategy, so AMANDA can't fail back to an incremental.
> 
> What I hoped would happen was that AMANDA would hit a timeout, fail  
> one of the DLEs, continue the other, and then restart the failed one  
> when the finished one was completed. What is happening (so far) is  
> both DLEs are just hanging.
> 
> Thanks,
> 
> Jordan
> 
> On Apr 28, 2008, at 11:09 AM, Brian Cuttler wrote:
> 
> >Unclear, you have two concurrent amdump tasks running or you have
> >two DLE's running concurrently (which is really the goal).
> >
> >For concurrent amdump tasks.
> >I've had that problem only when sharing my holding partition with
> >other jobs, either users or with competing amanda jobs (yes, I do
> >do some recommended things, detailed in older emails or ask me
> >offline). Separation of holding from user areas as well as staggering
> >of amdump jobs are the two steps needed in order to do this  
> >successfully.
> >
> >For a single amdump task:
> >I have the (strong) understanding that amanda will schedule DLEs
> >based on holding disk estimates. Are your estimates very far off ?
> >
> >Does amanda leave a percentage error margine for the estimates when
> >scheduling based on holding capacity ?
> >
> >On Mon, Apr 28, 2008 at 10:51:53AM -0400, Jordan Desroches wrote:
> >>Dear Amanda community,
> >>
> >>I am running two dumpers simultaneously, and using software
> >>compression. The estimates of the compression are not exact, and I've
> >>been running into a problem where the two DLEs AMANDA chooses to dump
> >>are bigger when combined than the holding disk. The holding disk  
> >>fills
> >>up before either dump finishes, and the dumps stop. In this  
> >>situation,
> >>what does AMANDA do? So far, the dumps have just hung there, but my
> >>timeouts are quite long. I could break the DLEs up into small chunks,
> >>but I was wondering if there is a more elegant solution.
> >>
> >>Thanks,
> >>
> >>Jordan
> >>
> >>-----------------------------------------------
> >>Jordan Desroches
> >>Systems Administrator
> >>Thayer School of Engineering at Dartmouth College
> >>8000 Cummings Hall
> >>Hanover, NH 03755
> >>
> >>phone: 603-646-8192
> >>fax:       603-646-9856
> >>
> >>
> >>
> >>
> >---
> >  Brian R Cuttler                 brian.cuttler AT wadsworth DOT org
> >  Computer Systems Support        (v) 518 486-1697
> >  Wadsworth Center                (f) 518 473-6384
> >  NYS Department of Health        Help Desk 518 473-0773
> >
> >
> >
> >IMPORTANT NOTICE: This e-mail and any attachments may contain
> >confidential or sensitive information which is, or may be, legally
> >privileged or otherwise protected by law from further disclosure.  It
> >is intended only for the addressee.  If you received this in error or
> >from someone who was not authorized to send it to you, please do not
> >distribute, copy or use it or any attachments.  Please notify the
> >sender immediately by reply e-mail and delete this from your
> >system. Thank you for your cooperation.
> >
> >
> 
> -----------------------------------------------
> Jordan Desroches
> Systems Administrator
> Thayer School of Engineering at Dartmouth College
> 8000 Cummings Hall
> Hanover, NH 03755
> 
> phone: 603-646-8192
> fax:       603-646-9856
> 
> 
> 
> 
---
   Brian R Cuttler                 brian.cuttler AT wadsworth DOT org
   Computer Systems Support        (v) 518 486-1697
   Wadsworth Center                (f) 518 473-6384
   NYS Department of Health        Help Desk 518 473-0773



IMPORTANT NOTICE: This e-mail and any attachments may contain
confidential or sensitive information which is, or may be, legally
privileged or otherwise protected by law from further disclosure.  It
is intended only for the addressee.  If you received this in error or
from someone who was not authorized to send it to you, please do not
distribute, copy or use it or any attachments.  Please notify the
sender immediately by reply e-mail and delete this from your
system. Thank you for your cooperation.