On 6/30/2011 4:41 PM, C. Ronoz wrote:
>
> Back-up is still running...
> root@artemis:~# ps aux | grep rsyn
> root      1674  8.8  1.4  19384  7240 ?        Ss   19:00  24:23 
> /usr/bin/rsync --server --sender --numeric-ids --perms --owner --group -D 
> --links --hard-links --times --block-size=2048 --recursive --ignore-times . /
> root      4726  0.0  0.1   7548   852 pts/2    S+   23:36   0:00 grep rsyn
> root@artemis:~# date
> Thu Jun 30 23:36:39 CEST 2011
>
> I don't understand how back-up could be so slow and at the same time provide 
> high cpu usage,
>   1800 backuppc  20   0 90320  33m 1308 R 78.9(% CPU)  3.4 178:52.82 
> BackupPC_dump

I think linux counts disk io in cpu use.

> On Thu, Jun 30, 2011 at 03:39:37PM -0500, Les Mikesell wrote:
>>
>> Running in a VM imposes a lot of overhead.  Running LVM on top of a file
>> based disk image pretty much guarantees that your disk block writes
>> won't be aligned with the physical disk which makes things much slower.
>>    Can you at least give the vm a real partition if that isn't one
>> already? And you definitely need to be sure you aren't sharing that
>> physical disk with anything else. More ram would probably help just by
>> providing more filesystem buffering even if you don't see it being used
>> otherwise.  You can turn off compression, but unless CPU capacity is the
>> problem it won't help and might make things worse due to more physical
>> disk activity.
>>
> I did not use LVM before repartitioning my back-up disk and it was the same 
> speed. People told me to use LVM, so I did. I will try to turn off 
> compression and see how this affects performance.

I haven't solved that particular problem myself.  I've seen others claim 
that LVM overhead is low by itself, but it seemed bad the few times I've 
used it in VM's - but I can't remember if I used a sparse image or not. 
That could have made it much worse by growing in non-contiguous segments.

I think the real issue is getting the block alignment to match the 
underlying physical disk with an appropriate offset to the logical 
partition start.  Is that a physical disk connected to the VM or is it a 
virtual image?

>> Backuppc will never be as fast as other systems, but the main situations
>> where the difference should be big are where you have a huge number of
>> small files (enough that the copy of the directory that is transferred
>> first pushes into swap) or when copying huge files with differences
>> where the server has to uncompress the existing copy and reconstruct it.
>>
>> After you have completed 2 fulls, you may see a speed increase on
>> unchanged files if you are using the --checksum-seed option.
>>
> Yes, I am aware that speed would improve after full back-up has been 
> completed because incremental back-ups only include 5% of the files or so. 
> How would BackupPC never be as fast as other systems? Because of 
> deduplication or?

BackupPC does more work with its rsync-in-perl implementation and 
compression than native rsync.  The de-dup overhead shouldn't be all 
that much in terms of cpu use, but it is going to involve a lot of 
directory lookups that may translate to head seeks and VM overhead if 
they aren't in cache.

> I am using a fairly regular BackupPC configuration file 
> (http://www.howtoforge.com/linux_backuppc) and really hope one of you guys 
> could help me find out why the performance is so poor and how I could improve 
> it.

If you can, try doing the same thing on a non-VM setup.  Your question 
may turn out to be more about VM tuning than backuppc itself.

-- 
   Les Mikesell
    lesmikesell AT gmail DOT com


------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security 
threats, fraudulent activity, and more. Splunk takes this data and makes 
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/