BackupPC-users

Re: [BackupPC-users] backup the backuppc pool with bacula

2009-06-11 12:02:16
Subject: Re: [BackupPC-users] backup the backuppc pool with bacula
From: Les Mikesell <lesmikesell AT gmail DOT com>
To: Adam Goryachev <mailinglists AT websitemanagers.com DOT au>
Date: Thu, 11 Jun 2009 10:57:26 -0500
Adam Goryachev wrote:
> 
>>> In fact, the POSIS/SUS specifications specifically states:
>>>    Some implementations mark for update the st_ctime field of renamed
>>>    files and some do not. Applications which make use of the st_ctime
>>>    field may behave differently with respect to renamed files unless they
>>>    are designed to allow for either behavior.
>>>
>>> However, it wouldn't be hard to add a "touch" to the chain renumbering
>>> routine if you want to be able to identify newly renumbered files. One
>>> would need to make sure that this doesn't have other unintended side
>>> effects but I don't think that BackupPC otherwise uses the file mtime.
>> Or, just do the explicit link/unlink operations to force the filesystem 
>> to do the right thing with ctime().
> 
> As long as the file you are dealing with has nlinks > 1 and those other
> files don't vanish in between the unlink/link.... rename is an atomic
> operation... unlink + link is not.

But that doesn't matter in this case (and it's link/unlink or you lose 
it).  You are working with the pool file name - and you don't really 
want the contents related to that name to atomically change without 
anything else knowing about it anyway.  Originally, backups weren't 
permitted at the same time as the nightly run to avoid that.  Now there 
must be some kind of locking.

>> And I'd like a quick/cheap way so you could just ignore the pool during 
>> a copy and rebuild it the same way it was built in the first place 
>> without thinking twice.  And maybe do things like backing up other 
>> instances of backuppc archives ignoring their pools and merging them so 
>> you could restore individual files directly.
> 
> Would that mean your data transfer is equal to the un-pooled size
> though? ie, if you transfer a single pc/<hostname> directory with 20
> full backups, you would need to transfer 20 X size of full backup of
> data. When it gets to the other side, you simply add the files from the
> first full backup to the pool, and then throw away (and link) the other
> 19 copies.

I'm not sure if rsync figures out the linked copies on the sending side 
or not.  It at least seems possible, and I've always been able to rsync 
any single pc tree.  Tar would only send one, but it includes one 
instance in each run so each incremental would repeat files you already 
have.

> Adds simplicity, but does it pose a problem with data sizes being
> transferred ?
> 
> One optimisation would be to examine the backuppc log, and only send the
> files that are not "same" or some such...

Some sort of client/server protocol would be needed to get it completely 
right.

-- 
   Les Mikesell
    lesmikesell AT gmail DOT com


------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables unlimited
royalty-free distribution of the report engine for externally facing 
server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/