Bacula-users

[Bacula-users] Bacula didn't start anymore

2009-05-26 11:09:56
Subject: [Bacula-users] Bacula didn't start anymore
From: ElmarRath AT systemcompetence DOT de
To: bacula-users AT lists.sourceforge DOT net
Date: Tue, 26 May 2009 17:01:09 +0200

Haloo,

I hope someone could help me wirth my problem. Monday morning faild my backup with the following log.



25-May 02:01 bocsles-dir JobId 124: Start Backup JobId 124, Job=Nightly-Save-bocsles.2009-05-25_02.01.00.02
25-May 02:01 bocsles-dir JobId 124: Using Device "LTO4"
25-May 02:01 bocsles-sd JobId 124: Error: 25-May 02:01 bocsles-sd JobId 124: Volume "LIN-TAG-22" previously written, moving to end of data.
25-May 02:01 bocsles-sd JobId 124: Ready to append to end of Volume "LIN-TAG-22" at file=136.
bocsles-fd JobId 124:      /dev is a different filesystem. Will not descend from / into /dev
bocsles-fd JobId 124:      /opt is a different filesystem. Will not descend from / into /opt
bocsles-fd JobId 124:      /var/lib/ntp/proc is a different filesystem. Will not descend from / into /var/lib/ntp/proc
bocsles-fd JobId 124:      /home is a different filesystem. Will not descend from / into /home
bocsles-fd JobId 124:      /var/lib/ntp/proc is a different filesystem. Will not descend from /var into /var/lib/ntp/proc
25-May 02:54 bocsles-sd JobId 124: Job write elapsed time = 00:52:53, Transfer rate = 22.19 M bytes/second
25-May 03:15 bocsles-dir JobId 124: Fatal error: Can't fill File table Query failed: INSERT INTO File (FileIndex, JobId, PathId, FilenameId, LStat, MD5)SELECT batch.FileIndex, batch.JobId, Path.PathId, Filename.FilenameId,batch.LStat, batch.MD5 FROM batch JOIN Path ON (batch.Path = Path.Path) JOIN Filename ON (batch.Name = Filename.Name): ERR=disk I/O error

25-May 03:15 bocsles-dir JobId 124: Error: sql_update.c:178 sql_update.c:178 update UPDATE Job SET JobStatus='f',EndTime='2009-05-25 03:15:26',ClientId=1,JobBytes=70287992114,JobFiles=805529,JobErrors=0,VolSessionId=1,VolSessionTime=1243209010,PoolId=3,FileSetId=3,JobTDate=1243214126,RealEndTime='2009-05-25 03:15:26',PriorJobId=0 WHERE JobId=124 failed:
disk I/O error
25-May 03:15 bocsles-dir JobId 124: sql_update.c:178 UPDATE Job SET JobStatus='f',EndTime='2009-05-25 03:15:26',ClientId=1,JobBytes=70287992114,JobFiles=805529,JobErrors=0,VolSessionId=1,VolSessionTime=1243209010,PoolId=3,FileSetId=3,JobTDate=1243214126,RealEndTime='2009-05-25 03:15:26',PriorJobId=0 WHERE JobId=124
25-May 03:15 bocsles-dir JobId 124: Warning: Error updating job record. sql_update.c:178 update UPDATE Job SET JobStatus='f',EndTime='2009-05-25 03:15:26',ClientId=1,JobBytes=70287992114,JobFiles=805529,JobErrors=0,VolSessionId=1,VolSessionTime=1243209010,PoolId=3,FileSetId=3,JobTDate=1243214126,RealEndTime='2009-05-25 03:15:26',PriorJobId=0 WHERE JobId=124 failed:
disk I/O error
25-May 03:15 bocsles-dir JobId 124: Fatal error: sql_get.c:289 sql_get.c:289 query SELECT VolSessionId,VolSessionTime,PoolId,StartTime,EndTime,JobFiles,JobBytes,JobTDate,Job,JobStatus,Type,Level,ClientId,Name,PriorJobId,RealEndTime,JobId,FileSetId,SchedTime,RealEndTime FROM Job WHERE JobId=124 failed:
disk I/O error
25-May 03:15 bocsles-dir JobId 124: sql_get.c:289 SELECT VolSessionId,VolSessionTime,PoolId,StartTime,EndTime,JobFiles,JobBytes,JobTDate,Job,JobStatus,Type,Level,ClientId,Name,PriorJobId,RealEndTime,JobId,FileSetId,SchedTime,RealEndTime FROM Job WHERE JobId=124
25-May 03:15 bocsles-dir JobId 124: Warning: Error getting Job record for Job report: ERR=sql_get.c:289 query SELECT VolSessionId,VolSessionTime,PoolId,StartTime,EndTime,JobFiles,JobBytes,JobTDate,Job,JobStatus,Type,Level,ClientId,Name,PriorJobId,RealEndTime,JobId,FileSetId,SchedTime,RealEndTime FROM Job WHERE JobId=124 failed:
disk I/O error
25-May 03:15 bocsles-dir JobId 124: Fatal error: sql_get.c:672 sql_get.c:672 query SELECT ClientId,Name,Uname,AutoPrune,FileRetention,JobRetention FROM Client WHERE Client.Name='bocsles-fd' failed:
disk I/O error
25-May 03:15 bocsles-dir JobId 124: sql_get.c:672 SELECT ClientId,Name,Uname,AutoPrune,FileRetention,JobRetention FROM Client WHERE Client.Name='bocsles-fd'
25-May 03:15 bocsles-dir JobId 124: Warning: Error getting Client record for Job report: ERR=Client record not found in Catalog.
25-May 03:15 bocsles-dir JobId 124: Fatal error: sql_get.c:942 sql_get.c:942 query SELECT MediaId,VolumeName,VolJobs,VolFiles,VolBlocks,VolBytes,VolMounts,VolErrors,VolWrites,MaxVolBytes,VolCapacityBytes,MediaType,VolStatus,PoolId,VolRetention,VolUseDuration,MaxVolJobs,MaxVolFiles,Recycle,Slot,FirstWritten,LastWritten,InChanger,EndFile,EndBlock,VolParts,LabelType,LabelDate,StorageId,Enabled,LocationId,RecycleCount,InitialWrite,ScratchPoolId,RecyclePoolId,VolReadTime,VolWriteTime FROM Media WHERE VolumeName='LIN-TAG-22' failed:
disk I/O error
25-May 03:15 bocsles-dir JobId 124: sql_get.c:942 SELECT MediaId,VolumeName,VolJobs,VolFiles,VolBlocks,VolBytes,VolMounts,VolErrors,VolWrites,MaxVolBytes,VolCapacityBytes,MediaType,VolStatus,PoolId,VolRetention,VolUseDuration,MaxVolJobs,MaxVolFiles,Recycle,Slot,FirstWritten,LastWritten,InChanger,EndFile,EndBlock,VolParts,LabelType,LabelDate,StorageId,Enabled,LocationId,RecycleCount,InitialWrite,ScratchPoolId,RecyclePoolId,VolReadTime,VolWriteTime FROM Media WHERE VolumeName='LIN-TAG-22'
25-May 03:15 bocsles-dir JobId 124: Warning: Error getting Media record for Volume "LIN-TAG-22": ERR=Media record for Vol=LIN-TAG-22 not found in Catalog.
25-May 03:15 bocsles-dir JobId 124: Fatal error: sql_get.c:359 sql_get.c:359 query SELECT VolumeName,MAX(VolIndex) FROM JobMedia,Media WHERE JobMedia.JobId=124 AND JobMedia.MediaId=Media.MediaId GROUP BY VolumeName ORDER BY 2 ASC failed:
disk I/O error
25-May 03:15 bocsles-dir JobId 124: sql_get.c:359 SELECT VolumeName,MAX(VolIndex) FROM JobMedia,Media WHERE JobMedia.JobId=124 AND JobMedia.MediaId=Media.MediaId GROUP BY VolumeName ORDER BY 2 ASC
25-May 03:15 bocsles-dir JobId 124: Error: Bacula bocsles-dir 2.4.4 (28Dec08): 25-May-2009 03:15:26
 Build OS:               powerpc64-unknown-linux-gnu suse 10
 JobId:                  124
 Job:                    Nightly-Save-bocsles.2009-05-25_02.01.00.02
 Backup Level:           Full
 Client:                 "bocsles-fd"
 FileSet:                "BOC_Full_Set" 2009-03-17 02:01:00
 Pool:                   "TagesPool" (From Job resource)
 Storage:                "LTO4" (From Job resource)
 Scheduled time:         25-May-2009 02:01:00
 Start time:             25-May-2009 02:01:03
 End time:               25-May-2009 03:15:26
 Elapsed time:           1 hour 14 mins 23 secs
 Priority:               10
 FD Files Written:       805,529
 SD Files Written:       805,529
 FD Bytes Written:       70,287,992,114 (70.28 GB)
 SD Bytes Written:       70,412,688,427 (70.41 GB)
 Rate:                   15749.0 KB/s
 Software Compression:   None
 VSS:                    no
 Storage Encryption:     no
 Volume name(s):        
 Volume Session Id:      1
 Volume Session Time:    1243209010
 Last Volume Bytes:      0 (0 B)
 Non-fatal FD errors:    1
 SD Errors:              0
 FD termination status:  OK
 SD termination status:  OK
 Termination:            *** Backup Error ***

25-May 03:15 bocsles-dir JobId 124: Begin pruning Jobs.
25-May 03:15 bocsles-dir JobId 124: Fatal error: sql_create.c:488 sql_create.c:488 query SELECT ClientId,Uname FROM Client WHERE Name='bocsles-fd' failed:
disk I/O error
25-May 03:15 bocsles-dir JobId 124: sql_create.c:488 SELECT ClientId,Uname FROM Client WHERE Name='bocsles-fd'
25-May 03:15 bocsles-dir JobId 124: Fatal error: sql_create.c:523 sql_create.c:523 insert INSERT INTO Client (Name,Uname,AutoPrune,FileRetention,JobRetention) VALUES ('bocsles-fd','',0,0,0) failed:
disk I/O error
25-May 03:15 bocsles-dir JobId 124: sql_create.c:523 INSERT INTO Client (Name,Uname,AutoPrune,FileRetention,JobRetention) VALUES ('bocsles-fd','',0,0,0)
25-May 03:15 bocsles-dir JobId 124: Error: sql_create.c:525 Create DB Client record INSERT INTO Client (Name,Uname,AutoPrune,FileRetention,JobRetention) VALUES ('bocsles-fd','',0,0,0) failed. ERR=disk I/O error
25-May 03:15 bocsles-dir JobId 124: Begin pruning Files.
25-May 03:15 bocsles-dir JobId 124: Fatal error: sql_create.c:488 sql_create.c:488 query SELECT ClientId,Uname FROM Client WHERE Name='bocsles-fd' failed:
disk I/O error
25-May 03:15 bocsles-dir JobId 124: sql_create.c:488 SELECT ClientId,Uname FROM Client WHERE Name='bocsles-fd'
25-May 03:15 bocsles-dir JobId 124: Fatal error: sql_create.c:523 sql_create.c:523 insert INSERT INTO Client (Name,Uname,AutoPrune,FileRetention,JobRetention) VALUES ('bocsles-fd','',0,0,0) failed:
disk I/O error
25-May 03:15 bocsles-dir JobId 124: sql_create.c:523 INSERT INTO Client (Name,Uname,AutoPrune,FileRetention,JobRetention) VALUES ('bocsles-fd','',0,0,0)
25-May 03:15 bocsles-dir JobId 124: Error: sql_create.c:525 Create DB Client record INSERT INTO Client (Name,Uname,AutoPrune,FileRetention,JobRetention) VALUES ('bocsles-fd','',0,0,0) failed. ERR=disk I/O error
25-May 03:15 bocsles-dir JobId 124: End auto prune.




After that I tried to restart bacula and get now the following problem:


bocsles:~ # /root/bacula/bin/bacula-dir -t
bacula-dir: dird.c:878 Could not open Catalog "MyCatalog", database "bacula".
bacula-dir: dird.c:883 sqlite.c:210 Unable to open Database=/root/bacula/working/bacula.db. ERR=out of memory
26-May 16:54 bacula-dir ERROR TERMINATION
Please correct configuration file: bacula-dir.conf
bocsles:~ #

I don't know why it thinks, that there is no memory left. I checked my harddrives and there was more then 14GB space left. Also I didn't have any problems with my swap or RAM, there should be more then enough memory left for the backup job to run.

BTW: The bacula.db has a size of 2 GB, could that be a problem for SQLite?

I hope you could help me.

Best regards


Mit freundlichen Gruessen


Wolf-Elmar Rath

SYSTEM COMPETENCE GmbH
Ohlendorfer Strasse 40
21220 Seevetal

Telefon: 04185-77-0
Telefax: 04185-77-440
Email: elmar.rath AT systemcompetence DOT de

Sitz der Gesellschaft: Seevetal
Geschäftsführer: Sven Hannemann, Martin Pranzner, Rigo Rothenbröker
Handelsregister: HRB 111223, Amtsgericht Lüneburg
------------------------------------------------------------------------------
Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, & 
iPhoneDevCamp asthey present alongside digital heavyweights like Barbarian
Group, R/GA, & Big Spaceship. http://www.creativitycat.com 
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users