Hi,
Adam Goryachev wrote on 2009-06-03 10:29:34 +1000 [Re: [BackupPC-users] Backing
up a BackupPC server]:
> [...]
> Might I (re)-propose the following:
> [...]
> 2) Extend backuppc-link so that after it links a file to the pool, or
> replaces a file in the pc directory with a link to the pool, then it
> will also add an entry to a DB (use the perl-DBI interface to make it
> generic or similar). Add some code to backuppc_nightly to modify the
> database when it modifies the pool as well.
> [...]
> Next, write a tool that can read the database + pool, and create the
> hardlinks under the pc directory. (So after you copy the pool, you can
> re-create the hardlinks).
>
> Thus, for people you want to copy their entire backuppc pool/etc, they
> can do it with a database + overhead, and everyone else can continue
> as-is.
I don't want to disappoint you, but that is trivial (to implement) even
without modifying BackupPC. If I'm allowed to write to a database, I can,
with a few lines of Perl code,
1. Create a table "pool" mapping inode numbers to pool files
(that's a simple iteration over the pool adding (inode, path) pairs)
2. Create a second table "pc" mapping pc/ path names to inode numbers
(that's a simple iteration over pc/ adding (path, inode) pairs)
Your pc-directory-re-creation tool now reads something like
SELECT pool.path AS src, pc.path AS dst
FROM pc, pool
WHERE pc.inode = pool.inode
ORDER BY pc.path
and does "link ($src, $dest);" after creating any directories $dest might
need (would ordering by pc.inode work better?). Yes, that statement is going
to produce an *enormous* result table. Let's just hope the database is smart
enough to handle that. If not, you can always do a
SELECT pool.path
FROM pool
WHERE pool.inode = ?
and execute that for each inode number you find in pc/ ... you won't even need
the "pc" table then. But still: the "pool" table will be large. The size of
this table is what determines when rsync et al. will stop working. Let's hope
an index on pool.inode will make the DB work wonders. We're still accessing the
information in random inode order, so we're not likely to get (m)any benefits
from caching. No, this is probably not the way to go.
Obviously, the same is true if this information is added to the database
during BackupPC_link operation and maintained by BackupPC_nightly.
> I'm sure the above doesn't handle 100% of needed cases,
Same for my ideas.
If I'm *not* allowed to write to a database, I'll need slightly more brains,
but the difficult part is handled by a "sort" invocation. Sorting 1.1 GB took
2.5 minutes - considerably less than traversing the pool and pc directories.
Actually, the same approach can be used with a database and one single table,
but then the database is really doing nothing except sorting data and
supplying it one time in the correct order.
I like your idea of separating the pool copy operation from re-establishing
the pc/ links. That sort of eliminates all RPC-type problems from my original
task. I only need to figure out how to handle non-pooled content, but that's
probably only stuff immediately in the pc/$host directories and 0-byte files
(which aren't that hard to re-create on the target system ;-). That way, I
would actually end up with something like BackupPC_tarAllPCsCopy without the
tar part (and without the overhead of calculating all pool hashes).
Regards,
Holger
------------------------------------------------------------------------------
OpenSolaris 2009.06 is a cutting edge operating system for enterprises
looking to deploy the next generation of Solaris that includes the latest
innovations from Sun and the OpenSource community. Download a copy and
enjoy capabilities such as Networking, Storage and Virtualization.
Go to: http://p.sf.net/sfu/opensolaris-get
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
|
