Bacula-users

[Bacula-users] Catastrophic overflow block problems

2013-01-17 11:41:57
Subject: [Bacula-users] Catastrophic overflow block problems
From: Ruth Ivimey-Cook <ruth AT ivimey DOT org>
To: bacula-users AT lists.sourceforge DOT net
Date: Thu, 17 Jan 2013 16:06:26 +0000
Hi,

I am sometimes getting these errors in my bacula backups:
Fatal error: device.c:192 Catastrophic error. Cannot write overflow block to device "DiskStorage-drive-0" 
and it is more likely on the larger volume backups. It seemingly results from bacula trying to write an additional block to a disk drive that is already 100% full. How can I stop bacula from believing this is a valid thing to do?

Background:  I have bacula setup on my local network to backup a file server and a number of workstations. The file server is also the bacula director and is running Fedora 15 and "bacula-common-5.0.3-28.fc15.x86_64". Bacula is writing backups to an iSCSI disk group (not array) over ethernet; there are 6 disks of 1TB to 2TB size and these are managed using "vchanger" 0.8.6, with 6 magazines each with 24 virtual volumes. The file server has 3.5TB of files and other workstations add about another 1TB.

More-complete log:
17-Jan 14:49 helva-sd JobId 3417: Recycled volume "DiskPool1_0006_0017" on device "DiskStorage-drive-0" (/var/spool/bacula/vchanger/0/drive0), all previous data lost.
17-Jan 14:49 helva-sd JobId 3417: New volume "DiskPool1_0006_0017" mounted on device "DiskStorage-drive-0" (/var/spool/bacula/vchanger/0/drive0) at 17-Jan-2013 14:49.
17-Jan 14:49 helva-sd JobId 3417: End of Volume "DiskPool1_0006_0017" at 0:216 on device "DiskStorage-drive-0" (/var/spool/bacula/vchanger/0/drive0). Write of 64512 bytes got 3879.
17-Jan 14:49 helva-sd JobId 3417: End of medium on Volume "DiskPool1_0006_0017" Bytes=217 Blocks=0 at 17-Jan-2013 14:49.
17-Jan 14:49 helva-sd JobId 3417: 3307 Issuing autochanger "unload slot 89, drive 0" command.
17-Jan 14:49 helva-dir JobId 3417: Using Volume "DiskPool1_0006_0018" from 'Scratch' pool.
17-Jan 14:49 helva-sd JobId 3417: 3301 Issuing autochanger "loaded? drive 0" command.
17-Jan 14:49 helva-sd JobId 3417: 3302 Autochanger "loaded? drive 0", result: nothing loaded.
17-Jan 14:49 helva-sd JobId 3417: 3304 Issuing autochanger "load slot 90, drive 0" command.
17-Jan 14:49 helva-sd JobId 3417: 3305 Autochanger "load slot 90, drive 0", status is OK.
17-Jan 14:49 helva-sd JobId 3417: Recycled volume "DiskPool1_0006_0018" on device "DiskStorage-drive-0" (/var/spool/bacula/vchanger/0/drive0), all previous data lost.
17-Jan 14:49 helva-sd JobId 3417: New volume "DiskPool1_0006_0018" mounted on device "DiskStorage-drive-0" (/var/spool/bacula/vchanger/0/drive0) at 17-Jan-2013 14:49.
17-Jan 14:49 helva-sd JobId 3417: End of Volume "DiskPool1_0006_0018" at 0:216 on device "DiskStorage-drive-0" (/var/spool/bacula/vchanger/0/drive0). Write of 64512 bytes got 3879.
17-Jan 14:49 helva-sd JobId 3417: Fatal error: device.c:192 Catastrophic error. Cannot write overflow block to device "DiskStorage-drive-0" (/var/spool/bacula/vchanger/0/drive0). ERR=No space left on device17-Jan 14:49 helva-fd JobId 3417: Error: bsock.c:393 Write error sending 65562 bytes to Storage daemon:helva.cam.ivimey.org:9103: ERR=Connection reset by peer
17-Jan 14:49 helva-fd JobId 3417: Fatal error: backup.c:1024 Network send error to SD. ERR=Connection reset by peer
17-Jan 14:49 helva-sd JobId 3417: Fatal error: device.c:192 Catastrophic error. Cannot write overflow block to device "DiskStorage-drive-0" (/var/spool/bacula/vchanger/0/drive0). ERR=No space left on device17-Jan 14:49 helva-sd JobId 3417: Fatal error: device.c:192 Catastrophic error. Cannot write overflow block to device "DiskStorage-drive-0" (/var/spool/bacula/vchanger/0/drive0). ERR=No space left on device17-Jan 14:49 helva-sd JobId 3417: Fatal error: device.c:192 Catastrophic error. Cannot write overflow block to device "DiskStorage-drive-0" (/var/spool/bacula/vchanger/0/drive0). ERR=No space left on device17-Jan 14:49 helva-sd JobId 3417: Fatal error: device.c:192 Catastrophic error. Cannot write overflow block to device "DiskStorage-drive-0" (/var/spool/bacula/vchanger/0/drive0). ERR=No space left on device17-Jan 14:49 helva-sd JobId 3417: Job write elapsed time = 14:30:55, Transfer rate = 12.06 M Bytes/second
17-Jan 14:49 helva-dir JobId 3417: Error: Bacula helva-dir 5.0.3 (04Aug10): 17-Jan-2013 14:49:32
  Build OS:               x86_64-redhat-linux-gnu redhat 
  JobId:                  3417
  Job:                    Helva_Home.2013-01-17_00.17.26_23
  Backup Level:           Full
  Client:                 "helva-fd" 5.0.3 (04Aug10) x86_64-redhat-linux-gnu,redhat,
  FileSet:                "Home" 2010-12-07 13:37:32
  Pool:                   "Normal-Full-18w" (From Job FullPool override)
  Catalog:                "MyCatalog" (From Client resource)
  Storage:                "DiskStorage" (From command line)
  Scheduled time:         17-Jan-2013 00:17:26
  Start time:             17-Jan-2013 00:17:28
  End time:               17-Jan-2013 14:49:32
  Elapsed time:           14 hours 32 mins 4 secs
  Priority:               12
  FD Files Written:       162,621
  SD Files Written:       162,621
  FD Bytes Written:       630,336,973,872 (630.3 GB)
  SD Bytes Written:       630,355,165,137 (630.3 GB)
  Rate:                   12046.8 KB/s
  Software Compression:   7.7 %
  VSS:                    no
  Encryption:             no
  Accurate:               no
  Volume name(s):         DiskPool1_0006_0003|DiskPool1_0001_0020|DiskPool1_0004_0013|DiskPool1_0004_0014|DiskPool1_0004_0023|DiskPool1_0004_0024|DiskPool1_0006_0017|DiskPool1_0006_0018
  Volume Session Id:      417
  Volume Session Time:    1351720132
  Last Volume Bytes:      217 (217 B)
  Non-fatal FD errors:    1
  SD Errors:              1
  FD termination status:  Error
  SD termination status:  Error
  Termination:            *** Backup Error ***


-- 
Software Manager & Engineer
Blog: http://www.ivimey.org/blog
LinkedIn: http://uk.linkedin.com/in/ruthivimeycook/ 
------------------------------------------------------------------------------
Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
MVPs and experts. ON SALE this month only -- learn more at:
http://p.sf.net/sfu/learnmore_122712
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
<Prev in Thread] Current Thread [Next in Thread>