Bacula-users

Re: [Bacula-users] missing a file during verification

2012-02-06 12:00:22
Subject: Re: [Bacula-users] missing a file during verification
From: Martin Simmons <martin AT lispworks DOT com>
To: bacula-users AT lists.sourceforge DOT net
Date: Mon, 6 Feb 2012 16:57:48 GMT
>>>>> On Fri, 3 Feb 2012 14:34:22 -0800, Doug Sampson said:
> 
> > > >>>>> On Thu, 2 Feb 2012 13:11:15 -0800, Doug Sampson said:
> > > >
> > > > > >>>>> On Tue, 31 Jan 2012 11:02:59 -0800, Doug Sampson said:
> > > > > >
> > > > > > > >>>>> On Mon, 30 Jan 2012 13:28:47 -0800, Doug Sampson said:
> > > > > > > >
> > > > > > > > Hello-
> > > > > > > >
> > > > > > > > Since upgrading to 5.2.3 from 5.0.3 on a FreeBSD 7.4
> server
> > two
> > > > > > weeks
> > > > > > > > ago, I've started seeing this message:
> > > > > > > >
> > > > > > > > 28-Jan 06:08 pisces-sd JobId 8299: End of Volume at file 1
> > on
> > > > device
> > > > > > > > "FileStoragecepheus" (/backup), Volume
> > > > > > > > "Backup_cepheus-fd_2012-01-28_00.05.01"
> > > > > > > > 28-Jan 06:08 pisces-sd JobId 8299: End of all volumes.
> > > > > > > > 28-Jan 06:08 pisces-dir JobId 8299: Error: Bacula
> pisces-dir
> > > > 5.2.3
> > > > > > > > (16Dec11):
> > > > > > > >   Build OS:               i386-portbld-freebsd7.4 freebsd
> > > > > > 7.4-RELEASE-p3
> > > > > > > >   JobId:                  8299
> > > > > > > >   Job:
> > > > cepheusVerifyData.2012-01-28_06.00.00_15
> > > > > > > >   FileSet:                cepheus
> > > > > > > >   Verify Level:           VolumeToCatalog
> > > > > > > >   Client:                 cepheus-fd
> > > > > > > >   Verify JobId:           8295
> > > > > > > >   Verify Job:
> > > > > > > >   Start time:             28-Jan-2012 06:00:00
> > > > > > > >   End time:               28-Jan-2012 06:08:40
> > > > > > > >   Files Expected:         989
> > > > > > > >   Files Examined:         988
> > > > > > > >   Non-fatal FD errors:    0
> > > > > > > >   FD termination status:  OK
> > > > > > > >   SD termination status:  OK
> > > > > > > >   Termination:            *** Verify Error ***
> > > > > > > >
> > > > > > > > This is for a differential backup of a Windows machine
> each
> > > > > > Saturday.
> > > > > > > > The daily incremental backups appear to be fine. The
> machine
> > > > > > 'cepheus'
> > > > > > > > is a Windows 2003 server running Winbacula 5.2.3. Prior to
> > > > upgrading
> > > > > > to
> > > > > > > > 5.2.3 on all Bacula devices, there were no errors reported
> > on
> > > > this
> > > > > > > > instance.
> > > > > > > >
> > > > > > > > How do I turn on or increase logging to try to identify
> the
> > > > reason
> > > > > > for
> > > > > > > > the skip? Or at least identify the file being skipped?
> > > > > > >
> > > > > > > Check the log lines above the ones you posted.  Bacula
> should
> > > > print
> > > > > > >
> > > > > > > The following files are in the Catalog but not on the
> > Volume(s):
> > > > > > >
> > > > > > > followed by the list of them.
> > > > > > >
> > > > > > > If not, what is your Message resource in the
> bacula-dir.conf?
> > > > > > >
> > > > > >
> > > > > > Whoops. I thought I had copied the entire message for JobId
> > 8299.
> > > > I'm
> > > > > > enclosing both JobId 8295 (the original backup job) and JobId
> > 8299
> > > > (the
> > > > > > verify job) here:
> > > > > >  ...
> > > > > > As you can see Bacula does not print anything indicating which
> > file
> > > > is
> > > > > > not on the Volume. Message resource info below:
> > > > > >  ...
> > > > > > Does these help?
> > > > >
> > > > > The messages look normal -- I would expect it to list them.
> > > > >
> > > > > You could try the following SQL commands to get more info about
> > what
> > > > is in
> > > > > the
> > > > > catalog for this job:
> > > > >
> > > > > SELECT COUNT(*) FROM File
> > > > >       WHERE File.JobId=8295;
> > > > >
> > > > > SELECT COUNT(*) FROM File
> > > > >       WHERE File.JobId=8295 AND File.FileIndex > 0;
> > > > >
> > > > > SELECT Path.Path,Filename.Name FROM File,Path,Filename
> > > > >       WHERE File.JobId=8295 AND File.FileIndex > 0
> > > > >       AND File.MarkId!=8299 AND File.PathId=Path.PathId
> > > > >       AND File.FilenameId=Filename.FilenameId;
> > > > >
> > > >
> > > >
> > > > $ psql bacula
> > > > Welcome to psql 8.3.16, the PostgreSQL interactive terminal.
> > > >
> > > > Type:  \copyright for distribution terms
> > > >        \h for help with SQL commands
> > > >        \? for help with psql commands
> > > >        \g or terminate with semicolon to execute query
> > > >        \q to quit
> > > >
> > > > bacula=# SELECT COUNT(*) FROM File
> > > > bacula-#       WHERE File.JobId=8295;
> > > >  count
> > > > -------
> > > >    988
> > > > (1 row)
> > > >
> > > > bacula=# SELECT COUNT(*) FROM File
> > > > bacula-#       WHERE File.JobId=8295 AND File.FileIndex > 0;
> > > >  count
> > > > -------
> > > >    988
> > > > (1 row)
> > > >
> > > > bacula=# SELECT Path.Path,Filename.Name FROM File,Path,Filename
> > > > bacula-#       WHERE File.JobId=8295 AND File.FileIndex > 0
> > > > bacula-#       AND File.MarkId!=8299 AND File.PathId=Path.PathId
> > > > bacula-#       AND File.FilenameId=Filename.FilenameId;
> > > >  path | name
> > > > ------+------
> > > > (0 rows)
> > > >
> > > > bacula=#
> > > >
> > > >
> > > > Um, so where's the error coming from?
> > > >
> > > > ~Doug
> > > >
> > >
> > > The error occurs because Bacula thinks there are 989 files in the
> > backup,
> > > though only 988 were recorded in the File table.
> > >
> > > What is the fileset definition?
> > 
> > # This is my cepheus (W2K3 Server) fileset:
> > FileSet {
> >   Name = "cepheus"
> >   enable VSS = "yes"
> >   Include {
> >     Options {
> >       signature = MD5
> > #      aclsupport = yes
> >       compression = GZIP6
> >   Exclude = yes
> >     IgnoreCase = yes
> >   # Exclude Mozilla-based programs' file caches
> >     WildDir = "[A-Z]:/Documents and Settings/*/Application
> > Data/*/Profiles/*/*/Cache"
> >     WildDir = "[A-Z]:/Documents and Settings/*/Application
> > Data/*/Profiles/*/*/Cache.Trash"
> >     WildDir = "[A-Z]:/Documents and Settings/*/Application
> > Data/*/Profiles/*/*/ImapMail"
> >   # Exclude user's registry files - they're always in use anyway.
> >     WildFile = "[A-Z]:/Documents and Settings/*/Local
> > Settings/Application Data/Microsoft/Windows/usrclass.*"
> >     WildFile = "[A-Z]:/Documents and Settings/*/ntuser.*"
> >   # Exclude directories full of lots and lots of useless little files
> >     WildDir = "[A-Z]:/Documents and Settings/*/Cookies"
> >     WildDir = "[A-Z]:/Documents and Settings/*/Recent"
> >     WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/History"
> >     WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/Temp"
> >     WildDir = "[A-Z]:/Documents and Settings/*/Local
> Settings/Temporary
> > Internet Files"
> >   # These are always open and unable to be backed up
> >     WildFile = "[A-Z]:/Documents and Settings/All Users/Application
> > Data/Microsoft/Network/Downloader/qmgr[01].dat"
> >   # Some random bits of Windows we want to ignore
> >     WildFile = "[A-Z]:/Windows/security/logs/scepol.log"
> >     WildDir = "[A-Z]:/Windows/system32/config"
> >     WildDir = "[A-Z]:/Windows/msdownld.tmp"
> >     WildDir = "[A-Z]:/Windows/Internet Logs"
> >     WildDir = "[A-Z]:/Windows/$Nt*Uninstall*"
> >     WildDir = "[A-Z]:/Windows/sysvol"
> >     WildFile = "[A-Z]:/Windows/cluster/CLUSDB"
> >     WildFile = "[A-Z]:/Windows/cluster/CLUSDB.LOG"
> >     WildFile = "[A-Z]:/Windows/NTDS/edb.log"
> >     WildFile = "[A-Z]:/Windows/NTDS/ntds.dit"
> >     WildFile = "[A-Z]:/Windows/NTDS/temp.edb"
> >     WildFile = "[A-Z]:/Windows/ntfrs/jet/log/edb.log"
> >     WildFile = "[A-Z]:/Windows/ntfrs/jet/ntfrs.jdb"
> >     WildFile = "[A-Z]:/Windows/ntfrs/jet/temp/tmp.edb"
> >     WildFile = "[A-Z]:/Windows/system32/CPL.CFG"
> >     WildFile = "[A-Z]:/Windows/system32/dhcp/dhcp.mdb"
> >     WildFile = "[A-Z]:/Windows/system32/dhcp/j50.log"
> >     WildFile = "[A-Z]:/Windows/system32/dhcp/tmp.edb"
> >     WildFile = "[A-Z]:/Windows/system32/LServer/edb.log"
> >     WildFile = "[A-Z]:/Windows/system32/LServer/TLSLic.edb"
> >     WildFile = "[A-Z]:/Windows/system32/LServer/tmp.edb"
> >     WildFile = "[A-Z]:/Windows/system32/wins/j50.log"
> >     WildFile = "[A-Z]:/Windows/system32/wins/wins.mdb"
> >     WildFile = "[A-Z]:/Windows/system32/wins/winstmp.mdb"
> >   # Temporary directories & files
> >     WildDir = "[A-Z]:/Windows/Temp"
> >     WildDir = "[A-Z]:/[Tt][Ee][Mm][Pp]"
> >     WildFile = "*.tmp"
> >     WildFile = "*.avi"
> >     WildFile = "*.mpg"
> >     WildFile = "*.mpeg"
> >     WildFile = "*.wmv"
> >     WildFile = "*.iso"
> >     WildFile = "*.rmvb"
> >     WildDir = "[A-Z]:/tmp"
> >     WildDir = "[A-Z]:/var/tmp"
> >   # Recycle bins
> >     WildDir = "[A-Z]:/RECYCLER"
> >   # Swap files
> >     WildFile = "[A-Z]:/pagefile.sys"
> >   # These are programs and are easier to reinstall than restore from
> > backup
> >     WildDir = "[A-Z]:/cygwin"
> >     WildDir = "[A-Z]:/Program Files/Grisoft"
> >     WildDir = "[A-Z]:/Program Files/Java"
> >     WildDir = "[A-Z]:/Program Files/Java Web Start"
> >     WildDir = "[A-Z]:/Program Files/JavaSoft"
> >     WildDir = "[A-Z]:/Program Files/Microsoft Office"
> >     WildDir = "[A-Z]:/Program Files/Mozilla Firefox"
> >     WildDir = "[A-Z]:/Program Files/Mozilla Thunderbird"
> >     WildDir = "[A-Z]:/Program Files/mozilla.org"
> >     WildDir = "[A-Z]:/Program Files/OpenOffice*"
> >     }
> >     Options {
> >       # Files that we don't want to compress too heavily.
> >       signature = MD5
> >       aclsupport = yes
> >       compression = GZIP1
> >       HFS Plus Support = yes
> >  # compressed archival formats
> >       wild = *.gz
> >       wild = *.gzip
> >       wild = *.tgz
> >       wild = *.tz
> >       wild = *.bz
> >       wild = *.bz2
> >       wild = *.bzip
> >       wild = *.bzip2
> >       wild = *.Z
> >       wild = *.7z
> >       wild = *.zip
> >       wild = *.zipx
> >       wild = *.iso
> >       wild = *.dmg
> >       wild = *.cab
> >       wild = *.rar
> >       wild = *.sea
> >       wild = *.sit
> >       wild = *.sitx
> > # compressed image formats
> >       wild = *.jpg
> >       wild = *.jpeg
> >       wild = *.gif
> >       wild = *.png
> >       wild = *.bmp
> >  # compressed video formats
> >       wild = *.dv
> >       wild = *.flv
> >       wild = *.m2v
> >       wild = *.mp2
> >       wild = *.mp4
> >       wild = *.mpeg
> >       wild = *.mpg
> >       wild = *.mov
> >       wild = *.qt
> >       wild = *.wmv
> >  # compressed audio formats
> >       wild = *.aac
> >       wild = *.mp3
> >     }
> >   # Cepheus has C: and D: as the main hard drives.
> >     File = "C:/"
> >     File = "D:/"
> >   }
> > }

OK, I don't see any problems with the fileset.


> > >
> > > What does the following SQL command print?
> > >
> > > SELECT MIN(FileIndex), MAX(FileIndex) FROM File WHERE JobId=8295;
> > >
> > 
> > bacula=# SELECT MIN(FileIndex), MAX(FileIndex) FROM File WHERE
> > JobId=8295;
> >  min | max
> > -----+-----
> >    1 | 988
> > (1 row)
> > 
> > bacula=#
> > 
> > > You could try to compare the list of files printed by
> > >
> > > list files jobid=8295
> > >
> > > with the output of the bls command for the volume containing that
> job
> > > (that
> > > will print the files from all jobs on the volume, so you will have
> to
> > > filter
> > > it).
> > 
> > How do I output the results of the bconsole 'list' command to a file?
> > I've outputted from the 'bls' command quite easily.
> > 

Use

@output filename
list files jobid=8295
@output


> I forgot to report that the bls output indicated 988 files. So the extra
> file should be in the output from the bconsole 'list' command. Correct?

No, I was expecting bls to show 989 files.  The bconsole 'list' command should
match the SQL commands because it uses the same data.

It might be something caused by VSS.  What is the output of

SELECT ObjectName, ObjectType, FileIndex
       FROM RestoreObject
       WHERE JobId=8295;

__Martin

------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users