Bacula-users

Re: [Bacula-users] missing a file during verification

2012-02-03 17:39:46
Subject: Re: [Bacula-users] missing a file during verification
From: "Doug Sampson" <dougs AT dawnsign DOT com>
To: "Doug Sampson" <dougs AT dawnsign DOT com>, "Martin Simmons" <martin AT lispworks DOT com>, <bacula-users AT lists.sourceforge DOT net>
Date: Fri, 3 Feb 2012 14:34:22 -0800
> > >>>>> On Thu, 2 Feb 2012 13:11:15 -0800, Doug Sampson said:
> > >
> > > > >>>>> On Tue, 31 Jan 2012 11:02:59 -0800, Doug Sampson said:
> > > > >
> > > > > > >>>>> On Mon, 30 Jan 2012 13:28:47 -0800, Doug Sampson said:
> > > > > > >
> > > > > > > Hello-
> > > > > > >
> > > > > > > Since upgrading to 5.2.3 from 5.0.3 on a FreeBSD 7.4
server
> two
> > > > > weeks
> > > > > > > ago, I've started seeing this message:
> > > > > > >
> > > > > > > 28-Jan 06:08 pisces-sd JobId 8299: End of Volume at file 1
> on
> > > device
> > > > > > > "FileStoragecepheus" (/backup), Volume
> > > > > > > "Backup_cepheus-fd_2012-01-28_00.05.01"
> > > > > > > 28-Jan 06:08 pisces-sd JobId 8299: End of all volumes.
> > > > > > > 28-Jan 06:08 pisces-dir JobId 8299: Error: Bacula
pisces-dir
> > > 5.2.3
> > > > > > > (16Dec11):
> > > > > > >   Build OS:               i386-portbld-freebsd7.4 freebsd
> > > > > 7.4-RELEASE-p3
> > > > > > >   JobId:                  8299
> > > > > > >   Job:
> > > cepheusVerifyData.2012-01-28_06.00.00_15
> > > > > > >   FileSet:                cepheus
> > > > > > >   Verify Level:           VolumeToCatalog
> > > > > > >   Client:                 cepheus-fd
> > > > > > >   Verify JobId:           8295
> > > > > > >   Verify Job:
> > > > > > >   Start time:             28-Jan-2012 06:00:00
> > > > > > >   End time:               28-Jan-2012 06:08:40
> > > > > > >   Files Expected:         989
> > > > > > >   Files Examined:         988
> > > > > > >   Non-fatal FD errors:    0
> > > > > > >   FD termination status:  OK
> > > > > > >   SD termination status:  OK
> > > > > > >   Termination:            *** Verify Error ***
> > > > > > >
> > > > > > > This is for a differential backup of a Windows machine
each
> > > > > Saturday.
> > > > > > > The daily incremental backups appear to be fine. The
machine
> > > > > 'cepheus'
> > > > > > > is a Windows 2003 server running Winbacula 5.2.3. Prior to
> > > upgrading
> > > > > to
> > > > > > > 5.2.3 on all Bacula devices, there were no errors reported
> on
> > > this
> > > > > > > instance.
> > > > > > >
> > > > > > > How do I turn on or increase logging to try to identify
the
> > > reason
> > > > > for
> > > > > > > the skip? Or at least identify the file being skipped?
> > > > > >
> > > > > > Check the log lines above the ones you posted.  Bacula
should
> > > print
> > > > > >
> > > > > > The following files are in the Catalog but not on the
> Volume(s):
> > > > > >
> > > > > > followed by the list of them.
> > > > > >
> > > > > > If not, what is your Message resource in the
bacula-dir.conf?
> > > > > >
> > > > >
> > > > > Whoops. I thought I had copied the entire message for JobId
> 8299.
> > > I'm
> > > > > enclosing both JobId 8295 (the original backup job) and JobId
> 8299
> > > (the
> > > > > verify job) here:
> > > > >  ...
> > > > > As you can see Bacula does not print anything indicating which
> file
> > > is
> > > > > not on the Volume. Message resource info below:
> > > > >  ...
> > > > > Does these help?
> > > >
> > > > The messages look normal -- I would expect it to list them.
> > > >
> > > > You could try the following SQL commands to get more info about
> what
> > > is in
> > > > the
> > > > catalog for this job:
> > > >
> > > > SELECT COUNT(*) FROM File
> > > >       WHERE File.JobId=8295;
> > > >
> > > > SELECT COUNT(*) FROM File
> > > >       WHERE File.JobId=8295 AND File.FileIndex > 0;
> > > >
> > > > SELECT Path.Path,Filename.Name FROM File,Path,Filename
> > > >       WHERE File.JobId=8295 AND File.FileIndex > 0
> > > >       AND File.MarkId!=8299 AND File.PathId=Path.PathId
> > > >       AND File.FilenameId=Filename.FilenameId;
> > > >
> > >
> > >
> > > $ psql bacula
> > > Welcome to psql 8.3.16, the PostgreSQL interactive terminal.
> > >
> > > Type:  \copyright for distribution terms
> > >        \h for help with SQL commands
> > >        \? for help with psql commands
> > >        \g or terminate with semicolon to execute query
> > >        \q to quit
> > >
> > > bacula=# SELECT COUNT(*) FROM File
> > > bacula-#       WHERE File.JobId=8295;
> > >  count
> > > -------
> > >    988
> > > (1 row)
> > >
> > > bacula=# SELECT COUNT(*) FROM File
> > > bacula-#       WHERE File.JobId=8295 AND File.FileIndex > 0;
> > >  count
> > > -------
> > >    988
> > > (1 row)
> > >
> > > bacula=# SELECT Path.Path,Filename.Name FROM File,Path,Filename
> > > bacula-#       WHERE File.JobId=8295 AND File.FileIndex > 0
> > > bacula-#       AND File.MarkId!=8299 AND File.PathId=Path.PathId
> > > bacula-#       AND File.FilenameId=Filename.FilenameId;
> > >  path | name
> > > ------+------
> > > (0 rows)
> > >
> > > bacula=#
> > >
> > >
> > > Um, so where's the error coming from?
> > >
> > > ~Doug
> > >
> >
> > The error occurs because Bacula thinks there are 989 files in the
> backup,
> > though only 988 were recorded in the File table.
> >
> > What is the fileset definition?
> 
> # This is my cepheus (W2K3 Server) fileset:
> FileSet {
>   Name = "cepheus"
>   enable VSS = "yes"
>   Include {
>     Options {
>       signature = MD5
> #      aclsupport = yes
>       compression = GZIP6
>   Exclude = yes
>     IgnoreCase = yes
>   # Exclude Mozilla-based programs' file caches
>     WildDir = "[A-Z]:/Documents and Settings/*/Application
> Data/*/Profiles/*/*/Cache"
>     WildDir = "[A-Z]:/Documents and Settings/*/Application
> Data/*/Profiles/*/*/Cache.Trash"
>     WildDir = "[A-Z]:/Documents and Settings/*/Application
> Data/*/Profiles/*/*/ImapMail"
>   # Exclude user's registry files - they're always in use anyway.
>     WildFile = "[A-Z]:/Documents and Settings/*/Local
> Settings/Application Data/Microsoft/Windows/usrclass.*"
>     WildFile = "[A-Z]:/Documents and Settings/*/ntuser.*"
>   # Exclude directories full of lots and lots of useless little files
>     WildDir = "[A-Z]:/Documents and Settings/*/Cookies"
>     WildDir = "[A-Z]:/Documents and Settings/*/Recent"
>     WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/History"
>     WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/Temp"
>     WildDir = "[A-Z]:/Documents and Settings/*/Local
Settings/Temporary
> Internet Files"
>   # These are always open and unable to be backed up
>     WildFile = "[A-Z]:/Documents and Settings/All Users/Application
> Data/Microsoft/Network/Downloader/qmgr[01].dat"
>   # Some random bits of Windows we want to ignore
>     WildFile = "[A-Z]:/Windows/security/logs/scepol.log"
>     WildDir = "[A-Z]:/Windows/system32/config"
>     WildDir = "[A-Z]:/Windows/msdownld.tmp"
>     WildDir = "[A-Z]:/Windows/Internet Logs"
>     WildDir = "[A-Z]:/Windows/$Nt*Uninstall*"
>     WildDir = "[A-Z]:/Windows/sysvol"
>     WildFile = "[A-Z]:/Windows/cluster/CLUSDB"
>     WildFile = "[A-Z]:/Windows/cluster/CLUSDB.LOG"
>     WildFile = "[A-Z]:/Windows/NTDS/edb.log"
>     WildFile = "[A-Z]:/Windows/NTDS/ntds.dit"
>     WildFile = "[A-Z]:/Windows/NTDS/temp.edb"
>     WildFile = "[A-Z]:/Windows/ntfrs/jet/log/edb.log"
>     WildFile = "[A-Z]:/Windows/ntfrs/jet/ntfrs.jdb"
>     WildFile = "[A-Z]:/Windows/ntfrs/jet/temp/tmp.edb"
>     WildFile = "[A-Z]:/Windows/system32/CPL.CFG"
>     WildFile = "[A-Z]:/Windows/system32/dhcp/dhcp.mdb"
>     WildFile = "[A-Z]:/Windows/system32/dhcp/j50.log"
>     WildFile = "[A-Z]:/Windows/system32/dhcp/tmp.edb"
>     WildFile = "[A-Z]:/Windows/system32/LServer/edb.log"
>     WildFile = "[A-Z]:/Windows/system32/LServer/TLSLic.edb"
>     WildFile = "[A-Z]:/Windows/system32/LServer/tmp.edb"
>     WildFile = "[A-Z]:/Windows/system32/wins/j50.log"
>     WildFile = "[A-Z]:/Windows/system32/wins/wins.mdb"
>     WildFile = "[A-Z]:/Windows/system32/wins/winstmp.mdb"
>   # Temporary directories & files
>     WildDir = "[A-Z]:/Windows/Temp"
>     WildDir = "[A-Z]:/[Tt][Ee][Mm][Pp]"
>     WildFile = "*.tmp"
>     WildFile = "*.avi"
>     WildFile = "*.mpg"
>     WildFile = "*.mpeg"
>     WildFile = "*.wmv"
>     WildFile = "*.iso"
>     WildFile = "*.rmvb"
>     WildDir = "[A-Z]:/tmp"
>     WildDir = "[A-Z]:/var/tmp"
>   # Recycle bins
>     WildDir = "[A-Z]:/RECYCLER"
>   # Swap files
>     WildFile = "[A-Z]:/pagefile.sys"
>   # These are programs and are easier to reinstall than restore from
> backup
>     WildDir = "[A-Z]:/cygwin"
>     WildDir = "[A-Z]:/Program Files/Grisoft"
>     WildDir = "[A-Z]:/Program Files/Java"
>     WildDir = "[A-Z]:/Program Files/Java Web Start"
>     WildDir = "[A-Z]:/Program Files/JavaSoft"
>     WildDir = "[A-Z]:/Program Files/Microsoft Office"
>     WildDir = "[A-Z]:/Program Files/Mozilla Firefox"
>     WildDir = "[A-Z]:/Program Files/Mozilla Thunderbird"
>     WildDir = "[A-Z]:/Program Files/mozilla.org"
>     WildDir = "[A-Z]:/Program Files/OpenOffice*"
>     }
>     Options {
>       # Files that we don't want to compress too heavily.
>       signature = MD5
>       aclsupport = yes
>       compression = GZIP1
>       HFS Plus Support = yes
>  # compressed archival formats
>       wild = *.gz
>       wild = *.gzip
>       wild = *.tgz
>       wild = *.tz
>       wild = *.bz
>       wild = *.bz2
>       wild = *.bzip
>       wild = *.bzip2
>       wild = *.Z
>       wild = *.7z
>       wild = *.zip
>       wild = *.zipx
>       wild = *.iso
>       wild = *.dmg
>       wild = *.cab
>       wild = *.rar
>       wild = *.sea
>       wild = *.sit
>       wild = *.sitx
> # compressed image formats
>       wild = *.jpg
>       wild = *.jpeg
>       wild = *.gif
>       wild = *.png
>       wild = *.bmp
>  # compressed video formats
>       wild = *.dv
>       wild = *.flv
>       wild = *.m2v
>       wild = *.mp2
>       wild = *.mp4
>       wild = *.mpeg
>       wild = *.mpg
>       wild = *.mov
>       wild = *.qt
>       wild = *.wmv
>  # compressed audio formats
>       wild = *.aac
>       wild = *.mp3
>     }
>   # Cepheus has C: and D: as the main hard drives.
>     File = "C:/"
>     File = "D:/"
>   }
> }
> 
> >
> > What does the following SQL command print?
> >
> > SELECT MIN(FileIndex), MAX(FileIndex) FROM File WHERE JobId=8295;
> >
> 
> bacula=# SELECT MIN(FileIndex), MAX(FileIndex) FROM File WHERE
> JobId=8295;
>  min | max
> -----+-----
>    1 | 988
> (1 row)
> 
> bacula=#
> 
> > You could try to compare the list of files printed by
> >
> > list files jobid=8295
> >
> > with the output of the bls command for the volume containing that
job
> > (that
> > will print the files from all jobs on the volume, so you will have
to
> > filter
> > it).
> 
> How do I output the results of the bconsole 'list' command to a file?
> I've outputted from the 'bls' command quite easily.
> 

I forgot to report that the bls output indicated 988 files. So the extra
file should be in the output from the bconsole 'list' command. Correct?

~Doug

------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users