Bacula-users

Re: [Bacula-users] get_file_record want 1 got rows=2 after migration from MySQL to PostgreSQL

2010-10-20 10:44:33
Subject: Re: [Bacula-users] get_file_record want 1 got rows=2 after migration from MySQL to PostgreSQL
From: Robert Oschwald <roos AT symentis DOT com>
To: bacula-users AT lists.sourceforge DOT net
Date: Wed, 20 Oct 2010 16:41:16 +0200
Martin, thanks for your suggestions.

I now did a test:
 - Noted the last fileid in db.
 - Ran BackupCatalog job. In the file table, one entry was added for this job 
(JobID=3366):

select * from file where fileid > 273350941;
273350942;1;3366;7656;46696;0;"P0B stOS IGg B Bk Bl A FS1p6i BAA qZXI BMvvaQ 
BMvveo BMvveo A A C";"IIuhxgZwyUMitZ4C4p5m4g"

 - Then I ran the duplicate to tape job.
One thing I note is that the copy job is running (JobId=3367), and the 
BackupCatalog job is also again running (JobID=3368) when I start the copy job.

After copy, I checked the file entries again:
select * from file where fileid > 273350941;
273350942;1;3366;7656;46696;0;"P0B stOS IGg B Bk Bl A FS1p6i BAA qZXI BMvvaQ 
BMvveo BMvveo A A C";"IIuhxgZwyUMitZ4C4p5m4g"
273350943;1;3368;7656;46696;0;"P0B stOS IGg B Bk Bl A FS1p6i BAA qZXI BMvvaQ 
BMvveo BMvveo A A C";"IIuhxgZwyUMitZ4C4p5m4g"

As you can see, one entry was added for JobID 3368.

So copy seems not to be the cause of the problem.


Robert



Am 19.10.2010 um 16:57 schrieb Martin Simmons:

> I don't know if it the duplicates are caused by the copy job, but there are
> two interesting differences between the rows:
> 
> 1) The atime (access time) field in the LStat is different (BMvLg8 vs BMvJgV),
>   suggesting that the file's stat was read more than once.
> 
> 2) The FileIndex values are different (128385 vs 86).  These values increment
>   from 1 at the start of the job, so it seems that the order of the files in
>   the jobs is not very similar.
> 
> I think you need to do some tests, maybe periodically recording the value of
> 
> select max(fileid) from file;
> 
> to try to discover when the rows are added.
> 
> __Martin
> 
> 
> 
> 
>>>>>> On Tue, 19 Oct 2010 14:28:24 +0200, Robert Oschwald said:
>> 
>> Today I checked again the migrated pg82 db for last nights incremental of 
>> one of the servers.
>> Still the same error  for this backup's new entries when I click on one of 
>> the nodes in the bat restore select tree:
>> 
>> sql_get.c:156-0 === Problem!  sql_get.c:155 get_file_record want 1 got 
>> rows=2 PathId=138243 FilenameId=2533203
>> 
>> DB entry check:
>> SELECT * FROM public.file WHERE  file.filenameid = 2533203 AND pathid=138243 
>> and jobid = 3347;
>> 
>> Result:
>> FileId       FileIndex       JobId   PathId  FilenameId      MarkId  LStat   
>> MD5
>> 273044923 128385         3347 138243  2533203           0  "P0G VIFW IGk B A 
>> A A IB BAA I BMvLg8 BMvJIy BMvJIy A A E"  "3eBz2t04SgJGcJFHVq62Mw"
>> 273041096 86             3347 138243  2533203           0  "P0G VIFW IGk B A 
>> A A IB BAA I BMvJgV BMvJIy BMvJIy A A E"  "3eBz2t04SgJGcJFHVq62Mw"
>> 
>> I then checked my old Bacula 5.0.2 MySQL Database.
>> Here is a query for one of the oldest backups I have in the former mysql db:
>> 
>> SELECT * FROM File WHERE  filenameid = 42066 AND pathid=102145 and jobid = 
>> 2941;
>> 
>> FileId              FileIndex JobId  PathId    FilenameId MarkId  LStat      
>> MD5
>> 228247316    1910509 2941  102145  42066         0           P0C OhMx IGk B 
>> A A A J8 BAA I BMiHlI BDIcWc BLkzrq A A E  SiJb2FBj/EYO/nmnkj9i7w
>> 229942749    3605942 2941  102145  42066         0           P0C OhMx IGk B 
>> A A A J8 BAA I BMkXTF BDIcWc BLkzrq A A E  SiJb2FBj/EYO/nmnkj9i7w
>> 
>> Even in the former MySQL DB I got two entries for each file in the db.
>> 
>> Maybe this is related to the copy job for the "Default" Pool which copies to 
>> LTO-4 tapes (B2D2T) after all backups to the Default (disk) pool?
>> 
>> Pool {
>>  Name = Default
>>  Pool Type = Backup
>>  Storage = File
>>  Recycle = yes
>>  AutoPrune = yes
>>  Volume Retention = 14 days
>>  LabelFormat = b2d-
>>  Maximum Volume Bytes = 50G
>>  # Duplicate: Next pool to use if running copy jobs
>>  NextPool = Tape
>> }
>> 
>> Note that the error message is only displayed if:
>> - You start bacula-dir manually in the console or
>> - Add 2>&1 > log_file to the init.d start() line
>> 
>> e.g.:
>> daemon /usr/sbin/bacula-dir $2 ${DIR_OPTIONS} -c /etc/bacula/bacula-dir.conf 
>> 2>&1 > /var/log/bacula_startup.log
>> 
>> on CentOS5.
>> 
>> Any ideas?
>> 
>> Robert
>> 
>> 
>>>>>>>> On Mon, 18 Oct 2010 09:54:28 +0200, Robert Oschwald said:
>>>> 
>>>> I migrated Bacula 5.0.2 from MySQL to Bacula 5.0.3 PostgreSQL 8.1, 
>>>> and then migrated to PostgreSQL 8.2 using dump/restore.
>>>> ...snip...
>>>> Migration from PostgreSQL 8.1 to 8.2:
>>>> pg_dumpall > pg_dump.sql
>>>> - install postgreSQL 8.2 (CentOS-Testing)
>>>> psql < pg_dump.sql
>>> 
>>> Did you start with an empty database when you imported the dump after
>>> migration from PostgreSQL 8.1 to 8.2?  I suspect not, because you have
>>> everything twice.
>>> 
>>> 
>>>> 
>>>> 
>>>> Are these error messages critical?
>>> 
>>> Yes.
>>> 
>>> __Martin
>>> 
>>> ------------------------------------------------------------------------------
>>> Download new Adobe(R) Flash(R) Builder(TM) 4
>>> The new Adobe(R) Flex(R) 4 and Flash(R) Builder(TM) 4 (formerly 
>>> Flex(R) Builder(TM)) enable the development of rich applications that run
>>> across multiple browsers and platforms. Download your free trials today!
>>> http://p.sf.net/sfu/adobe-dev2dev
>>> _______________________________________________
>>> Bacula-users mailing list
>>> Bacula-users AT lists.sourceforge DOT net
>>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>> 
>> 
>> ------------------------------------------------------------------------------
>> Download new Adobe(R) Flash(R) Builder(TM) 4
>> The new Adobe(R) Flex(R) 4 and Flash(R) Builder(TM) 4 (formerly 
>> Flex(R) Builder(TM)) enable the development of rich applications that run
>> across multiple browsers and platforms. Download your free trials today!
>> http://p.sf.net/sfu/adobe-dev2dev
>> _______________________________________________
>> Bacula-users mailing list
>> Bacula-users AT lists.sourceforge DOT net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>> 
> 
> ------------------------------------------------------------------------------
> Download new Adobe(R) Flash(R) Builder(TM) 4
> The new Adobe(R) Flex(R) 4 and Flash(R) Builder(TM) 4 (formerly 
> Flex(R) Builder(TM)) enable the development of rich applications that run
> across multiple browsers and platforms. Download your free trials today!
> http://p.sf.net/sfu/adobe-dev2dev
> _______________________________________________
> Bacula-users mailing list
> Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users


------------------------------------------------------------------------------
Nokia and AT&T present the 2010 Calling All Innovators-North America contest
Create new apps & games for the Nokia N8 for consumers in  U.S. and Canada
$10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing
Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store 
http://p.sf.net/sfu/nokia-dev2dev
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users