-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Christoph Litauer wrote:
> Craig Barratt schrieb:
>> Christoph writes:
>>
>>> If I take a look on the structure of incrementals, I can see lots of
>>> empty directories. It seems as if the whole directory structure of the
>>> backup-source is kind of "duplicated" to the backup disk - although most
>>> of the directories (and the files in them) are unchanged.
>>> This leads to _lots_ of files/directories on the backuppc-disk (about 20
>>> million now). Is it necessary?
>> Yes - the directory structure needs to be complete, even for
>> an incremental. The storage used should be small.
>
> Craig,
>
> can you explain why, please?
>
> You're right: The storage amount is very small. But one can get _very_
> large directory structures on the backup filesystem. My BackupPC volume
> now uses 147,650,611 inodes in an XFS filesystem. (I think) this leads
> to a very slow directory creation:
> time for i in `seq 1 10000`; do mkdir $i; done
> runs about 2.5 minutes! This is 66 directories per second, whereas the
> same command on the same server but another (empty) xfs filesystem took
> only 34 seconds (about 5 times faster).
>
> Let's take a look at one (out of 10) of the backup clients:
> A level 0 dump consists of 285,207 directories and 1,140,363 files.
> A level 5 dump consists of 285,041 directories and 94,449 files.
> This level 5 dump contains 192069 emtpy directories.
>
> I am afraid of expanding my backup scheme ... more clients and longer
> incremental keep periods will lead to lots of millions of empty
> directories ...
I don't want to get into a war about filesystem formats, but perhaps
this is a valid data point for XFS. I don't know about other filesystem
types either...
You might like to check out/talk to some XFS experts, and see what they
say about your very slow performance.... There may be some options to
tune/improve the performance, or they may simply suggest another FS
format which is better suited to the workload in backuppc.
Either way, let us all know the results, I'm sure you aren't the only
person on the list with this issue.
BTW, I use reiserfs on my backuppc server, and will let you know the
number of directories/files under the backuppc directory, along with the
time to create new directories .... will probably take some hours for
the results :)
Regards,
Adam
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFIYR3bGyoxogrTyiURAlKEAKCGJlw8aaMaNF/jfJtG99zhpT/XYgCfaP/5
aR6ZCUVcONdl9MKSMbLoOYw=
=x8LR
-----END PGP SIGNATURE-----
-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
|
