BackupPC-users

Re: [BackupPC-users] speed up backups

2010-05-30 14:01:28
Subject: Re: [BackupPC-users] speed up backups
From: Les Mikesell <lesmikesell AT gmail DOT com>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Sun, 30 May 2010 12:59:07 -0500
Ralf Gross wrote:
> Ralf Gross schrieb:
>> Les Mikesell schrieb:
>>> On 5/26/2010 3:41 PM, Ralf Gross wrote:
>>>> Ralf Gross schrieb:
>>>>> write(1, "N\2\0\7\5\3lvs\r\0\0\0\r\0\0\0lvmiopversion8\5"..., 594) = 594
>>>>> select(1, [0], [], NULL, {60, 0})       = 0 (Timeout)
>>>>> select(1, [0], [], NULL, {60, 0})       = 0 (Timeout)
>>>>> select(1, [0], [], NULL, {60, 0})       = 0 (Timeout)
>>>>> select(1, [0], [], NULL, {60, 0})       = 0 (Timeout)
>>>>> select(1, [0], [], NULL, {60, 0}
>>>>>
>>>>>
>>>>> smells like a time out, but I don't know where. I found a couple of 
>>>>> messages
>>>>> with similar output in the list archives, but none of them had a solution 
>>>>> yet.
>>>> *grr*
>>>>
>>>> I only traced the Xfer PID, not the PID. BackupPC_dump seems to be
>>>> active and comparing the file list with the pool and I see high cpu
>>>> load.
>>>>
>>>> I'm sure that I haven't seen that as I abortet the backup before.
>>>> Now I'll have will wait until tomorrow morning...
>>> Until the 2nd full completes, the server side has to uncompress the 
>>> stored copy to compute the checkums on existing files.  And there may be 
>>> some quirk about switching from tar to rsync that I've forgotten.  Maybe 
>>> the 1st run will add the checksum cache for files you already have.
>>
>> The full rsync is still running sind 5/26 21:00. I'll report back when
>> it's done. 
> 
> 
> Ok, the first rsync full backup (488) completed. It took 500min. longer than
> the last tar full backup (482).
> 
> Backup        Type    Filled  Level   Start Date      Duration/mins  Age/days
> 482   full    yes     0       5/19 02:05      3223.2         11.5
> 483   incr    no      1       5/21 07:49        89.6          9.2
> 484   incr    no      2       5/22 03:05       136.4          8.4
> 485   incr    no      3       5/23 03:05       119.1          7.4
> 486   incr    no      4       5/24 03:05       111.4          6.4
> 487   incr    no      1       5/25 03:05       165.9          5.4
> 488   full    yes     0       5/26 21:00      3744.2          3.7
> 489   incr    no      1       5/29 12:15       394.1          1.1
> 490   incr    no      2       5/30 03:05       190.8          0.4
> 
> I'm not sure if the checksum caching will compensate this in after the 3rd
> backup. Anything else I could do to tune rsync?
> 

You could force a full to start on Friday evening so weekly scheduling will 
keep 
the full runs on weekends if they take more than a night to complete.  
Depending 
on how much daily change you have, you might want to set incremental levels for 
the intermediate runs.

A more extreme change would be to edit Rsync.pm to not add the --ignore-times 
option on fulls.  I haven't needed this myself yet but I think it would make a 
big difference in speed - at the expense of not checking files for unlikely but 
possible differences.

-- 
   Les Mikesell
    lesmikesell AT gmail DOT com

------------------------------------------------------------------------------

_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/