Bacula-users

Re: [Bacula-users] get_file_record want 1 got rows=2 after migration from MySQL to PostgreSQL

2010-10-20 14:34:13
Subject: Re: [Bacula-users] get_file_record want 1 got rows=2 after migration from MySQL to PostgreSQL
From: Martin Simmons <martin AT lispworks DOT com>
To: bacula-users AT lists.sourceforge DOT net
Date: Wed, 20 Oct 2010 19:30:51 +0100
Indeed, the copy job is an administative job, which creates a second job that
is the copy.  The latest select output shows the same FileIndex and LStat
atime for both jobs as expected.

__Martin


>>>>> On Wed, 20 Oct 2010 16:41:16 +0200, Robert Oschwald said:
> 
> Martin, thanks for your suggestions.
> 
> I now did a test:
>  - Noted the last fileid in db.
>  - Ran BackupCatalog job. In the file table, one entry was added for this job 
> (JobID=3366):
> 
> select * from file where fileid > 273350941;
> 273350942;1;3366;7656;46696;0;"P0B stOS IGg B Bk Bl A FS1p6i BAA qZXI BMvvaQ 
> BMvveo BMvveo A A C";"IIuhxgZwyUMitZ4C4p5m4g"
> 
>  - Then I ran the duplicate to tape job.
> One thing I note is that the copy job is running (JobId=3367), and the 
> BackupCatalog job is also again running (JobID=3368) when I start the copy 
> job.
> 
> After copy, I checked the file entries again:
> select * from file where fileid > 273350941;
> 273350942;1;3366;7656;46696;0;"P0B stOS IGg B Bk Bl A FS1p6i BAA qZXI BMvvaQ 
> BMvveo BMvveo A A C";"IIuhxgZwyUMitZ4C4p5m4g"
> 273350943;1;3368;7656;46696;0;"P0B stOS IGg B Bk Bl A FS1p6i BAA qZXI BMvvaQ 
> BMvveo BMvveo A A C";"IIuhxgZwyUMitZ4C4p5m4g"
> 
> As you can see, one entry was added for JobID 3368.
> 
> So copy seems not to be the cause of the problem.
> 
> 
> Robert
> 
> 
> 
> Am 19.10.2010 um 16:57 schrieb Martin Simmons:
> 
> > I don't know if it the duplicates are caused by the copy job, but there are
> > two interesting differences between the rows:
> > 
> > 1) The atime (access time) field in the LStat is different (BMvLg8 vs 
> > BMvJgV),
> >   suggesting that the file's stat was read more than once.
> > 
> > 2) The FileIndex values are different (128385 vs 86).  These values 
> > increment
> >   from 1 at the start of the job, so it seems that the order of the files in
> >   the jobs is not very similar.
> > 
> > I think you need to do some tests, maybe periodically recording the value of
> > 
> > select max(fileid) from file;
> > 
> > to try to discover when the rows are added.
> > 
> > __Martin
> > 
> > 
> > 
> > 
> >>>>>> On Tue, 19 Oct 2010 14:28:24 +0200, Robert Oschwald said:
> >> 
> >> Today I checked again the migrated pg82 db for last nights incremental of 
> >> one of the servers.
> >> Still the same error  for this backup's new entries when I click on one of 
> >> the nodes in the bat restore select tree:
> >> 
> >> sql_get.c:156-0 === Problem!  sql_get.c:155 get_file_record want 1 got 
> >> rows=2 PathId=138243 FilenameId=2533203
> >> 
> >> DB entry check:
> >> SELECT * FROM public.file WHERE  file.filenameid = 2533203 AND 
> >> pathid=138243 and jobid = 3347;
> >> 
> >> Result:
> >> FileId     FileIndex       JobId   PathId  FilenameId      MarkId  LStat   
> >> MD5
> >> 273044923 128385         3347 138243  2533203           0  "P0G VIFW IGk B 
> >> A A A IB BAA I BMvLg8 BMvJIy BMvJIy A A E"  "3eBz2t04SgJGcJFHVq62Mw"
> >> 273041096 86             3347 138243  2533203           0  "P0G VIFW IGk B 
> >> A A A IB BAA I BMvJgV BMvJIy BMvJIy A A E"  "3eBz2t04SgJGcJFHVq62Mw"
> >> 
> >> I then checked my old Bacula 5.0.2 MySQL Database.
> >> Here is a query for one of the oldest backups I have in the former mysql 
> >> db:
> >> 
> >> SELECT * FROM File WHERE  filenameid = 42066 AND pathid=102145 and jobid = 
> >> 2941;
> >> 
> >> FileId            FileIndex JobId  PathId    FilenameId MarkId  LStat      
> >> MD5
> >> 228247316  1910509 2941  102145  42066         0           P0C OhMx IGk B 
> >> A A A J8 BAA I BMiHlI BDIcWc BLkzrq A A E  SiJb2FBj/EYO/nmnkj9i7w
> >> 229942749  3605942 2941  102145  42066         0           P0C OhMx IGk B 
> >> A A A J8 BAA I BMkXTF BDIcWc BLkzrq A A E  SiJb2FBj/EYO/nmnkj9i7w
> >> 
> >> Even in the former MySQL DB I got two entries for each file in the db.
> >> 
> >> Maybe this is related to the copy job for the "Default" Pool which copies 
> >> to LTO-4 tapes (B2D2T) after all backups to the Default (disk) pool?
> >> 
> >> Pool {
> >>  Name = Default
> >>  Pool Type = Backup
> >>  Storage = File
> >>  Recycle = yes
> >>  AutoPrune = yes
> >>  Volume Retention = 14 days
> >>  LabelFormat = b2d-
> >>  Maximum Volume Bytes = 50G
> >>  # Duplicate: Next pool to use if running copy jobs
> >>  NextPool = Tape
> >> }
> >> 
> >> Note that the error message is only displayed if:
> >> - You start bacula-dir manually in the console or
> >> - Add 2>&1 > log_file to the init.d start() line
> >> 
> >> e.g.:
> >> daemon /usr/sbin/bacula-dir $2 ${DIR_OPTIONS} -c 
> >> /etc/bacula/bacula-dir.conf 2>&1 > /var/log/bacula_startup.log
> >> 
> >> on CentOS5.
> >> 
> >> Any ideas?
> >> 
> >> Robert
> >> 
> >> 
> >>>>>>>> On Mon, 18 Oct 2010 09:54:28 +0200, Robert Oschwald said:
> >>>> 
> >>>> I migrated Bacula 5.0.2 from MySQL to Bacula 5.0.3 PostgreSQL 8.1, 
> >>>> and then migrated to PostgreSQL 8.2 using dump/restore.
> >>>> ...snip...
> >>>> Migration from PostgreSQL 8.1 to 8.2:
> >>>> pg_dumpall > pg_dump.sql
> >>>> - install postgreSQL 8.2 (CentOS-Testing)
> >>>> psql < pg_dump.sql
> >>> 
> >>> Did you start with an empty database when you imported the dump after
> >>> migration from PostgreSQL 8.1 to 8.2?  I suspect not, because you have
> >>> everything twice.
> >>> 
> >>> 
> >>>> 
> >>>> 
> >>>> Are these error messages critical?
> >>> 
> >>> Yes.
> >>> 
> >>> __Martin
> >>> 
> >>> ------------------------------------------------------------------------------
> >>> Download new Adobe(R) Flash(R) Builder(TM) 4
> >>> The new Adobe(R) Flex(R) 4 and Flash(R) Builder(TM) 4 (formerly 
> >>> Flex(R) Builder(TM)) enable the development of rich applications that run
> >>> across multiple browsers and platforms. Download your free trials today!
> >>> http://p.sf.net/sfu/adobe-dev2dev
> >>> _______________________________________________
> >>> Bacula-users mailing list
> >>> Bacula-users AT lists.sourceforge DOT net
> >>> https://lists.sourceforge.net/lists/listinfo/bacula-users
> >> 
> >> 
> >> ------------------------------------------------------------------------------
> >> Download new Adobe(R) Flash(R) Builder(TM) 4
> >> The new Adobe(R) Flex(R) 4 and Flash(R) Builder(TM) 4 (formerly 
> >> Flex(R) Builder(TM)) enable the development of rich applications that run
> >> across multiple browsers and platforms. Download your free trials today!
> >> http://p.sf.net/sfu/adobe-dev2dev
> >> _______________________________________________
> >> Bacula-users mailing list
> >> Bacula-users AT lists.sourceforge DOT net
> >> https://lists.sourceforge.net/lists/listinfo/bacula-users
> >> 
> > 
> > ------------------------------------------------------------------------------
> > Download new Adobe(R) Flash(R) Builder(TM) 4
> > The new Adobe(R) Flex(R) 4 and Flash(R) Builder(TM) 4 (formerly 
> > Flex(R) Builder(TM)) enable the development of rich applications that run
> > across multiple browsers and platforms. Download your free trials today!
> > http://p.sf.net/sfu/adobe-dev2dev
> > _______________________________________________
> > Bacula-users mailing list
> > Bacula-users AT lists.sourceforge DOT net
> > https://lists.sourceforge.net/lists/listinfo/bacula-users
> 
> 
> ------------------------------------------------------------------------------
> Nokia and AT&T present the 2010 Calling All Innovators-North America contest
> Create new apps & games for the Nokia N8 for consumers in  U.S. and Canada
> $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing
> Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store 
> http://p.sf.net/sfu/nokia-dev2dev
> _______________________________________________
> Bacula-users mailing list
> Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
> 

------------------------------------------------------------------------------
Nokia and AT&T present the 2010 Calling All Innovators-North America contest
Create new apps & games for the Nokia N8 for consumers in  U.S. and Canada
$10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing
Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store 
http://p.sf.net/sfu/nokia-dev2dev
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users