Bacula-users

[Bacula-users] Full job terminates claiming no space left but tape gets more data

2012-12-10 08:24:05
Subject: [Bacula-users] Full job terminates claiming no space left but tape gets more data
From: "Robert Wirth" <Robert.Wirth AT dfki DOT de>
To: bacula-users AT lists.sourceforge DOT net
Date: Mon, 10 Dec 2012 14:04:51 +0100
Hi,

Situation:
----------
The backup strategy is doing a yearly Full, monthly Diff and daily Incr.
This worked well over years with Bacula 1.x and 2.x.  

Some months ago, I upgraded SD and DIR from Bacula 2.2.8 to 5.0.1
The Diff/Incr jobs continued to do well.

But now, while doing a Full for the first time under 5.0.1,
I got errors (and can reproduce them)

Hardware is two LTO-5 drives with a changer and a lot of tapes.

MaximumSpoolSize = 1024 GB
MaximumJobSpoolSize = 32 GB



What's up?
----------
I've 6 bigger jobs (TB size each) in the same pool and start them 
at the same time.  They're spooling and despooling alternately
to the same tape(s) as expected.

Tape AD0018L5 is written by the 6 jobs and, when full, 
automatically replaced by next free tape AD0016L5.  The 
latter one is also written by the 6 jobs...  

Then, one of the 6 jobs fails (and is rescheduled):

Committing spooled data to Volume "AD0016L5". Despooling 22,147,812,023 bytes 
...
Fseek on attributes file failed: ERR=No space left on device
...
Last Volume Bytes:      1,283,646,163,968 (1.283 TB)

Really, the tape isn't full!  The other 5 jobs (and this one 
after being rescheduled) continue to write that tape AD0016L5
which by now has 1,633,895,424,000 Bytes written and is still 
in Append state.


Question
--------
What may be the cause for that "ERR=No space left on device"

Same configuration worked well with Bacula <= 2.2.8 and LTO-3
(with lots more full tapes and changes during a Full session
than today when filling LTO-5)


More info:
----------

08-Dez 05:46 bup-serv-dir JobId 130936: Start Backup JobId 130936, 
Job=lnv-91163.2012-12-08_04.46.11_20
08-Dez 05:46 bup-serv-dir JobId 130936: Using Device "Tape1A"
08-Dez 05:46 bup-serv-sd JobId 130936: Spooling data ...
...
09-Dez 20:17 bup-serv-sd JobId 130936: Committing spooled data to Volume 
"AD0016L5". Despooling 22,147,812,023 bytes ...
10-Dez 00:52 bup-serv-sd JobId 130936: Despooling elapsed time = 00:34:51, 
Transfer rate = 10.59 M Bytes/second
10-Dez 00:52 bup-serv-sd JobId 130936: Fatal error: Fseek on attributes file 
failed: ERR=No space left on device
10-Dez 00:52 bup-serv-dir JobId 130936: Error: Bacula bup-serv-dir 5.0.1 
(24Feb10): 10-Dez-2012 00:52:50
...
  Elapsed time:           1 day 19 hours 6 mins 36 secs
  Priority:               11
  FD Files Written:       445,124
  SD Files Written:       445,124
  FD Bytes Written:       277,792,113,411 (277.7 GB)
  SD Bytes Written:       277,875,653,641 (277.8 GB)
...
  Volume name(s):         AD0018L5|AD0016L5
  Volume Session Id:      202
  Volume Session Time:    1354723843
  Last Volume Bytes:      1,283,646,163,968 (1.283 TB)
  Non-fatal FD errors:    0
  SD Errors:              0
  FD termination status:  OK
  SD termination status:  Error
  Termination:            *** Backup Error ***


list volumes pool=ServerFull
...
|   1,012 | AD0016L5   | Append    |       1 | 1,633,895,424,000 |    1,634 |   
63,072,000 |       1 |   56 |         1 | LTO-5     | 2012-12-10 10:24:20 |
...
|   1,014 | AD0018L5   | Full      |       1 | 1,896,767,502,336 |    1,897 |   
63,072,000 |       0 |   58 |         1 | LTO-5     | 2012-12-08 14:45:09 |


Regards, 

        Robert

---------------------------------------------------------------
+++   German Research Center for Artificial Intelligence    +++
---------------------------------------------------------------
Robert Wirth,Campus D32,Stuhlsatzenhausweg 3,66123 Saarbruecken
@phone:+49-681-85775-5078  Germany  mailto:Robert.Wirth AT dfki DOT de
---------------------------------------------------------------



------------------------------------------------------------------------------
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

<Prev in Thread] Current Thread [Next in Thread>