Bacula-users
[Top] [All Lists]
<prev [Date] next>
[Advanced]
 <prev [Thread] next>

Re: [Bacula-users] missing a file during verification

2012-02-03 16:51:41
Subject: Re: [Bacula-users] missing a file during verification
From: "Doug Sampson" <dougs AT dawnsign DOT com>
To: "Martin Simmons" <martin AT lispworks DOT com>, <bacula-users AT lists.sourceforge DOT net>
Date: Fri, 3 Feb 2012 13:47:51 -0800 
> >>>>> On Thu, 2 Feb 2012 13:11:15 -0800, Doug Sampson said:
> >
> > > >>>>> On Tue, 31 Jan 2012 11:02:59 -0800, Doug Sampson said:
> > > >
> > > > > >>>>> On Mon, 30 Jan 2012 13:28:47 -0800, Doug Sampson said:
> > > > > >
> > > > > > Hello-
> > > > > >
> > > > > > Since upgrading to 5.2.3 from 5.0.3 on a FreeBSD 7.4 server
two
> > > > weeks
> > > > > > ago, I've started seeing this message:
> > > > > >
> > > > > > 28-Jan 06:08 pisces-sd JobId 8299: End of Volume at file 1
on
> > device
> > > > > > "FileStoragecepheus" (/backup), Volume
> > > > > > "Backup_cepheus-fd_2012-01-28_00.05.01"
> > > > > > 28-Jan 06:08 pisces-sd JobId 8299: End of all volumes.
> > > > > > 28-Jan 06:08 pisces-dir JobId 8299: Error: Bacula pisces-dir
> > 5.2.3
> > > > > > (16Dec11):
> > > > > >   Build OS:               i386-portbld-freebsd7.4 freebsd
> > > > 7.4-RELEASE-p3
> > > > > >   JobId:                  8299
> > > > > >   Job:
> > cepheusVerifyData.2012-01-28_06.00.00_15
> > > > > >   FileSet:                cepheus
> > > > > >   Verify Level:           VolumeToCatalog
> > > > > >   Client:                 cepheus-fd
> > > > > >   Verify JobId:           8295
> > > > > >   Verify Job:
> > > > > >   Start time:             28-Jan-2012 06:00:00
> > > > > >   End time:               28-Jan-2012 06:08:40
> > > > > >   Files Expected:         989
> > > > > >   Files Examined:         988
> > > > > >   Non-fatal FD errors:    0
> > > > > >   FD termination status:  OK
> > > > > >   SD termination status:  OK
> > > > > >   Termination:            *** Verify Error ***
> > > > > >
> > > > > > This is for a differential backup of a Windows machine each
> > > > Saturday.
> > > > > > The daily incremental backups appear to be fine. The machine
> > > > 'cepheus'
> > > > > > is a Windows 2003 server running Winbacula 5.2.3. Prior to
> > upgrading
> > > > to
> > > > > > 5.2.3 on all Bacula devices, there were no errors reported
on
> > this
> > > > > > instance.
> > > > > >
> > > > > > How do I turn on or increase logging to try to identify the
> > reason
> > > > for
> > > > > > the skip? Or at least identify the file being skipped?
> > > > >
> > > > > Check the log lines above the ones you posted.  Bacula should
> > print
> > > > >
> > > > > The following files are in the Catalog but not on the
Volume(s):
> > > > >
> > > > > followed by the list of them.
> > > > >
> > > > > If not, what is your Message resource in the bacula-dir.conf?
> > > > >
> > > >
> > > > Whoops. I thought I had copied the entire message for JobId
8299.
> > I'm
> > > > enclosing both JobId 8295 (the original backup job) and JobId
8299
> > (the
> > > > verify job) here:
> > > >  ...
> > > > As you can see Bacula does not print anything indicating which
file
> > is
> > > > not on the Volume. Message resource info below:
> > > >  ...
> > > > Does these help?
> > >
> > > The messages look normal -- I would expect it to list them.
> > >
> > > You could try the following SQL commands to get more info about
what
> > is in
> > > the
> > > catalog for this job:
> > >
> > > SELECT COUNT(*) FROM File
> > >       WHERE File.JobId=8295;
> > >
> > > SELECT COUNT(*) FROM File
> > >       WHERE File.JobId=8295 AND File.FileIndex > 0;
> > >
> > > SELECT Path.Path,Filename.Name FROM File,Path,Filename
> > >       WHERE File.JobId=8295 AND File.FileIndex > 0
> > >       AND File.MarkId!=8299 AND File.PathId=Path.PathId
> > >       AND File.FilenameId=Filename.FilenameId;
> > >
> >
> >
> > $ psql bacula
> > Welcome to psql 8.3.16, the PostgreSQL interactive terminal.
> >
> > Type:  \copyright for distribution terms
> >        \h for help with SQL commands
> >        \? for help with psql commands
> >        \g or terminate with semicolon to execute query
> >        \q to quit
> >
> > bacula=# SELECT COUNT(*) FROM File
> > bacula-#       WHERE File.JobId=8295;
> >  count
> > -------
> >    988
> > (1 row)
> >
> > bacula=# SELECT COUNT(*) FROM File
> > bacula-#       WHERE File.JobId=8295 AND File.FileIndex > 0;
> >  count
> > -------
> >    988
> > (1 row)
> >
> > bacula=# SELECT Path.Path,Filename.Name FROM File,Path,Filename
> > bacula-#       WHERE File.JobId=8295 AND File.FileIndex > 0
> > bacula-#       AND File.MarkId!=8299 AND File.PathId=Path.PathId
> > bacula-#       AND File.FilenameId=Filename.FilenameId;
> >  path | name
> > ------+------
> > (0 rows)
> >
> > bacula=#
> >
> >
> > Um, so where's the error coming from?
> >
> > ~Doug
> >
> 
> The error occurs because Bacula thinks there are 989 files in the
backup,
> though only 988 were recorded in the File table.
> 
> What is the fileset definition?

# This is my cepheus (W2K3 Server) fileset: 
FileSet {
  Name = "cepheus"
  enable VSS = "yes"
  Include {
    Options {
      signature = MD5
#      aclsupport = yes
      compression = GZIP6
  Exclude = yes
    IgnoreCase = yes
  # Exclude Mozilla-based programs' file caches
    WildDir = "[A-Z]:/Documents and Settings/*/Application
Data/*/Profiles/*/*/Cache"
    WildDir = "[A-Z]:/Documents and Settings/*/Application
Data/*/Profiles/*/*/Cache.Trash"
    WildDir = "[A-Z]:/Documents and Settings/*/Application
Data/*/Profiles/*/*/ImapMail"
  # Exclude user's registry files - they're always in use anyway.
    WildFile = "[A-Z]:/Documents and Settings/*/Local
Settings/Application Data/Microsoft/Windows/usrclass.*"
    WildFile = "[A-Z]:/Documents and Settings/*/ntuser.*"
  # Exclude directories full of lots and lots of useless little files
    WildDir = "[A-Z]:/Documents and Settings/*/Cookies"
    WildDir = "[A-Z]:/Documents and Settings/*/Recent"
    WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/History"
    WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/Temp"
    WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/Temporary
Internet Files"
  # These are always open and unable to be backed up
    WildFile = "[A-Z]:/Documents and Settings/All Users/Application
Data/Microsoft/Network/Downloader/qmgr[01].dat"
  # Some random bits of Windows we want to ignore
    WildFile = "[A-Z]:/Windows/security/logs/scepol.log"
    WildDir = "[A-Z]:/Windows/system32/config"
    WildDir = "[A-Z]:/Windows/msdownld.tmp"
    WildDir = "[A-Z]:/Windows/Internet Logs"
    WildDir = "[A-Z]:/Windows/$Nt*Uninstall*"
    WildDir = "[A-Z]:/Windows/sysvol"
    WildFile = "[A-Z]:/Windows/cluster/CLUSDB"
    WildFile = "[A-Z]:/Windows/cluster/CLUSDB.LOG"
    WildFile = "[A-Z]:/Windows/NTDS/edb.log"
    WildFile = "[A-Z]:/Windows/NTDS/ntds.dit"
    WildFile = "[A-Z]:/Windows/NTDS/temp.edb"
    WildFile = "[A-Z]:/Windows/ntfrs/jet/log/edb.log"
    WildFile = "[A-Z]:/Windows/ntfrs/jet/ntfrs.jdb"
    WildFile = "[A-Z]:/Windows/ntfrs/jet/temp/tmp.edb"
    WildFile = "[A-Z]:/Windows/system32/CPL.CFG"
    WildFile = "[A-Z]:/Windows/system32/dhcp/dhcp.mdb"
    WildFile = "[A-Z]:/Windows/system32/dhcp/j50.log"
    WildFile = "[A-Z]:/Windows/system32/dhcp/tmp.edb"
    WildFile = "[A-Z]:/Windows/system32/LServer/edb.log"
    WildFile = "[A-Z]:/Windows/system32/LServer/TLSLic.edb"
    WildFile = "[A-Z]:/Windows/system32/LServer/tmp.edb"
    WildFile = "[A-Z]:/Windows/system32/wins/j50.log"
    WildFile = "[A-Z]:/Windows/system32/wins/wins.mdb"
    WildFile = "[A-Z]:/Windows/system32/wins/winstmp.mdb"
  # Temporary directories & files
    WildDir = "[A-Z]:/Windows/Temp"
    WildDir = "[A-Z]:/[Tt][Ee][Mm][Pp]"
    WildFile = "*.tmp"
    WildFile = "*.avi"
    WildFile = "*.mpg"
    WildFile = "*.mpeg"
    WildFile = "*.wmv"
    WildFile = "*.iso"
    WildFile = "*.rmvb"
    WildDir = "[A-Z]:/tmp"
    WildDir = "[A-Z]:/var/tmp"
  # Recycle bins 
    WildDir = "[A-Z]:/RECYCLER"
  # Swap files
    WildFile = "[A-Z]:/pagefile.sys"
  # These are programs and are easier to reinstall than restore from
backup
    WildDir = "[A-Z]:/cygwin" 
    WildDir = "[A-Z]:/Program Files/Grisoft"
    WildDir = "[A-Z]:/Program Files/Java"
    WildDir = "[A-Z]:/Program Files/Java Web Start"
    WildDir = "[A-Z]:/Program Files/JavaSoft"
    WildDir = "[A-Z]:/Program Files/Microsoft Office"
    WildDir = "[A-Z]:/Program Files/Mozilla Firefox"
    WildDir = "[A-Z]:/Program Files/Mozilla Thunderbird"
    WildDir = "[A-Z]:/Program Files/mozilla.org"
    WildDir = "[A-Z]:/Program Files/OpenOffice*"
    }
    Options {
      # Files that we don't want to compress too heavily.
      signature = MD5
      aclsupport = yes
      compression = GZIP1
      HFS Plus Support = yes
 # compressed archival formats
      wild = *.gz
      wild = *.gzip
      wild = *.tgz
      wild = *.tz
      wild = *.bz
      wild = *.bz2
      wild = *.bzip
      wild = *.bzip2
      wild = *.Z
      wild = *.7z
      wild = *.zip
      wild = *.zipx
      wild = *.iso
      wild = *.dmg
      wild = *.cab
      wild = *.rar     
      wild = *.sea
      wild = *.sit
      wild = *.sitx
# compressed image formats
      wild = *.jpg
      wild = *.jpeg
      wild = *.gif
      wild = *.png
      wild = *.bmp    
 # compressed video formats     
      wild = *.dv
      wild = *.flv
      wild = *.m2v
      wild = *.mp2 
      wild = *.mp4  
      wild = *.mpeg
      wild = *.mpg 
      wild = *.mov   
      wild = *.qt   
      wild = *.wmv  
 # compressed audio formats
      wild = *.aac
      wild = *.mp3
    }
  # Cepheus has C: and D: as the main hard drives.
    File = "C:/"
    File = "D:/"
  }
}

> 
> What does the following SQL command print?
> 
> SELECT MIN(FileIndex), MAX(FileIndex) FROM File WHERE JobId=8295;
> 

bacula=# SELECT MIN(FileIndex), MAX(FileIndex) FROM File WHERE
JobId=8295;
 min | max 
-----+-----
   1 | 988
(1 row)

bacula=#

> You could try to compare the list of files printed by
> 
> list files jobid=8295
> 
> with the output of the bls command for the volume containing that job
> (that
> will print the files from all jobs on the volume, so you will have to
> filter
> it).

How do I output the results of the bconsole 'list' command to a file?
I've outputted from the 'bls' command quite easily.

~Doug 

------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
[More with this subject...]
<Prev in Thread] Current Thread [Next in Thread>
Previous by Date:  Re: [Bacula-users] Archived tapes and pools question, John Drescher
Next by Date:  [Bacula-users] Recycling not respecting Volume Retention, Diego Rubert
Previous by Thread:  Re: [Bacula-users] missing a file during verification, Martin Simmons
Next by Thread:  Re: [Bacula-users] missing a file during verification, Doug Sampson
Indexes:  [Date] [Thread] [Top] [All Lists]