Christoph Litauer schrieb:
> Christoph Litauer schrieb:
>> Thanks a lot Adam!
>> In the meantime I discussed my problem on the xfs mailing list. We are
>> not finished yet, but adding mount option "nobarrier" reduced my
>> performance problems significantly. I am still in contact to clarify if
>> it's possible to optimize the usage of inode allocation groups. We will
>> see ...
>>
>> To get a better base for further discussions, I did a few benchmarks
>> using bonnie++:
>>
>> bonnie++ -u root -f -n 10:0:0:1000 -d /backuppc -s0
>>
>> result for file creation per second: 5501 (sequential)
>> result for file creation per second: 6430 (random)
>>
>> Without option nobarrier I had 137 files/second ..
>>
>
> Well ... this was the good news ...
> ... now the bad ones ...
>
> This morning, 2 backup processes were still running. strace'ing one
> client process, one can see that rsync is waiting for sending data
> (select). strace'ing the servers process (BackupPC_dump), one can see
> that it creates tons of directories ... which is correct as I have
> learned as the dump is an incremental one.
> But: BackupPC_dump only creates about 60 directories per second
> (regarding the number of used inodes). A bonnie++ benchmark _in
> parallel_ on backuppc's filesystem reports 1450 files/directories per
> second (minimum).
>
> Some of my clients have filesystems containig about 250,000 directories.
> So the creation of the directory tree lasts about 1.5 hours for each
> client plus the time for transfering the backup data. I think this maybe
> the reason why my backups are still running in the morning ...
>
> What are your directory creation rates regarding BackupPC_dump?
>
>
Well, as nobody could help me so far (even the xfs mailing list is very
slient ...), let's get a step further:
My filesystem for backuppc data now contains about 160 million inodes.
bonnie++ tells me that this filesystem is able to create 1000-4000
inodes per second. What I get in reality is about 50-150 inodes created
per second.
My scenario: I did an incremental BackupPC_dump of a small clients
filesystem containing about 70000 files in 5000 directories. Nearly no
data had changed on the client, so BackupPC just had to create the full
directory structure, about 5000 directories. While running,
BackupPC_dump consumed 100% cpu time, i/o wait was 0%. Watching the
creation rate of inodes, I got about 100 inodes/second.
I then started BackupPC_dump using Devel::Profiling as a profiling tool.
Result of dprofpp is:
Total Elapsed Time = 503.3859 Seconds
User+System Time = 116.8679 Seconds
Exclusive Times
%Time ExclSec CumulS #Calls sec/call Csec/c Name
13.7 16.06 70.642 5063 0.0032 0.0140 BackupPC::View::dirCache
8.33 9.735 27.133 30255 0.0003 0.0009 BackupPC::Attrib::read
7.66 8.950 19.010 33326 0.0003 0.0006 BackupPC::FileZIO::read
6.39 7.465 7.465 30378 0.0002 0.0002 BackupPC::Lib::dirRead
6.33 7.399 7.399 29709 0.0002 0.0002
Compress::Zlib::inflateStream::inflate
4.69 5.483 5.483 197001 0.0000 0.0000
BackupPC::Lib::fileNameEltMangle
3.97 4.643 4.643 186668 0.0000 0.0000
BackupPC::Lib::fileNameUnmangle
3.69 4.315 4.315 433 0.0100 0.0100
File::RsyncP::Digest::blockDigest
3.62 4.231 109.86 4 1.0578 27.466 File::RsyncP::fileCsumSend
3.57 4.176 78.075 76514 0.0001 0.0010
BackupPC::Xfer::RsyncFileIO::attribGetWhere
3.55 4.151 8.320 30419 0.0001 0.0003
BackupPC::Lib::fileNameMangle
3.32 3.880 7.436 30340 0.0001 0.0002 BackupPC::FileZIO::open
2.57 3.009 81.085 76514 0.0000 0.0011
BackupPC::Xfer::RsyncFileIO::attribGet
2.57 3.007 73.899 76514 0.0000 0.0010
BackupPC::Xfer::RsyncFileIO::viewCacheDir
2.25 2.634 2.634 76435 0.0000 0.0000 File::RsyncP::FileList::get
Maybe someone is able to interpret these results or perhaps use them as
a starting point for optimizations?
--
Regards
Christoph
________________________________________________________________________
Christoph Litauer litauer AT uni-koblenz DOT de
Uni Koblenz, Computing Center, http://www.uni-koblenz.de/~litauer
Postfach 201602, 56016 Koblenz Fon: +49 261 287-1311, Fax: -100 1311
PGP-Fingerprint: F39C E314 2650 650D 8092 9514 3A56 FBD8 79E3 27B2
-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
|
