Les Mikesell wrote at about 16:12:19 -0500 on Tuesday, June 2, 2009:
 > Craig Barratt wrote:
 > > 
 > > Yes, taking this approach would require a very substantial rewrite.
 > > BackupPC would become a lot simpler.  But it also creates a significant
 > > issue of backward compatibility.  The only solution would be to provide
 > > tools that import the old BackupPC store into a new one.  That is
 > > possible, but would likely be very slow.
 > 
 > One simple thing that I've sometimes thought would be useful would be a 
 > way to re-create the pool links down any PC tree.  That way you could 
 > rsync individual pc directories to an offsite location which usually 
 > works OK even with -H (I suppose there is some limit..) and then 
 > reconstruct the pooling to reclaim the duplicate space, repeating for 
 > each host or new backup without ever having to deal with the size of the 
 > combined pool/pc tree.
 > 

This would be a lot easier if the footer of the pool files included
the md5sum checksum name of each pool file. For compressed pool files,
this would just mean extending the footer by 16 bytes. Actually for
speed, it would probably be better to store this information in the
header after the initial magic byte. Even better add another couple of
bytes to identify which element of the chain is being referenced when
multiple files have the same pool hash (note: this would have to be
adjusted when BackupPC_nightly runs and re-arranges the chain
numbering).

Then you could pretty easily find the corresponding pool file from any
of the hard links without the usual reverse-lookup problem when trying
to identify the pool file from the hard-link inode in the pc file.

Backing up the BackupPC data would then be as simple as the following:
1. Shutdown BackupPC
2. Copy the pool to the new destination (no hard links)
3. Recurse through the pc directories as follows:
        - Copy directory entries to the new destination (i.e. recreate
          directories using something like mkdir)
        - Copy regular files with nlinks=1 to the new destination
        - For hard-linked files, use the header (or footer) to find the
          cpool pathname (reconstructed from the hash and the chain
          number). Then create the corresponding link on the new
          destination.
4. Restart BackupPC

If you don't add the pool hash information to the cpool file
header/footer, then you could still do a similar process by adding an
intermediate step (say 2.5) of creating a lookup table by recursing
through the pool and associating inodes with cpool entries. Then in
step 3 you would use the inode number of each hard-linked file in the
pc directory to look up the corresponding link that needs to be
created. This would require some cleverness to make the lookup fast
for large pools where the entire table might not fit into memory. My
only concern is that this may require O(n^2) or O(nlogn) operations
vs. the O(n) for the first method.



------------------------------------------------------------------------------
OpenSolaris 2009.06 is a cutting edge operating system for enterprises 
looking to deploy the next generation of Solaris that includes the latest 
innovations from Sun and the OpenSource community. Download a copy and 
enjoy capabilities such as Networking, Storage and Virtualization. 
Go to: http://p.sf.net/sfu/opensolaris-get
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/