BackupPC-users

Re: [BackupPC-users] Incremental directory structure

2008-06-24 11:57:16
Subject: Re: [BackupPC-users] Incremental directory structure
From: Christoph Litauer <litauer AT uni-koblenz DOT de>
To: Craig Barratt <cbarratt AT users.sourceforge DOT net>
Date: Tue, 24 Jun 2008 17:21:56 +0200
Craig Barratt schrieb:
> Christoph writes:
> 
>> If I take a look on the structure of incrementals, I can see lots of
>> empty directories. It seems as if the whole directory structure of the
>> backup-source is kind of "duplicated" to the backup disk - although most
>> of the directories (and the files in them) are unchanged.
>> This leads to _lots_ of files/directories on the backuppc-disk (about 20
>> million now). Is it necessary?
> 
> Yes - the directory structure needs to be complete, even for
> an incremental.  The storage used should be small.

Craig,

can you explain why, please?

You're right: The storage amount is very small. But one can get _very_
large directory structures on the backup filesystem. My BackupPC volume
now uses 147,650,611 inodes in an XFS filesystem. (I think) this leads
to a very slow directory creation:
    time for i in `seq 1 10000`; do mkdir $i; done
runs about 2.5 minutes! This is 66 directories per second, whereas the
same command on the same server but another (empty) xfs filesystem took
only 34 seconds (about 5 times faster).

Let's take a look at one (out of 10) of the backup clients:
A level 0 dump consists of 285,207 directories and 1,140,363 files.
A level 5 dump consists of 285,041 directories and    94,449 files.
This level 5 dump contains 192069 emtpy directories.

I am afraid of expanding my backup scheme ... more clients and longer 
incremental keep periods will lead to lots of millions of empty 
directories ...

-- 
Regards
Christoph
________________________________________________________________________
Christoph Litauer                  litauer AT uni-koblenz DOT de
Uni Koblenz, Computing Center,     http://www.uni-koblenz.de/~litauer
Postfach 201602, 56016 Koblenz     Fon: +49 261 287-1311, Fax: -100 1311
PGP-Fingerprint: F39C E314 2650 650D 8092 9514 3A56 FBD8 79E3 27B2


-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/