Hi,

Jeffrey J. Kosowsky wrote on 2009-06-03 19:36:22 -0400 [Re: [BackupPC-users] 
Backing up a BackupPC server]:
> [...]
> Holger, one thing I don't understand is that if you create a dump
> table associating inodes with pool file hashes, aren't we back in the
> same situation as using rsync -H?

in short (because it's once again really late): I sort the table by inode
numbers. The lines are constructed so that I first get the pool entry for one
inode number and then all the links to it. I only need to keep one inode
number and one pool file name in memory, because as soon as I reach the next
inode number, I know I won't need the information again.

For sorting, I simply use sort(1). I believe that's merge sort (O(n log n)).

At some point, it would be faster to create two independent tables and sort
them individually, like you point out (it may already be, but sorting is fast
enough so far that I don't worry about that yet). You'd then read the two
files in parallel, effectively doing much the same as with a single file.

> I would be curious to know how how in the real world the time (and
> memory usage) compares to copy over a large (say multi Terabyte)
> BackupPC topdir varies for the following methods:
> 
> 1. cp -ad
> 2. rsync -H
> 3. Copy using a single table of pool inode numbers
> 4. Copy using a sorted table of pool inode numbers and pc hierarchy
>    inode numbers

So would I, but I haven't got the resources at hand to test it.

Regards,
Holger

P.S.: I'm not sure creating the pool in the resulting order is optimal, but at
      least it appears to work.

------------------------------------------------------------------------------
OpenSolaris 2009.06 is a cutting edge operating system for enterprises 
looking to deploy the next generation of Solaris that includes the latest 
innovations from Sun and the OpenSource community. Download a copy and 
enjoy capabilities such as Networking, Storage and Virtualization. 
Go to: http://p.sf.net/sfu/opensolaris-get
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/