|
Re: [BackupPC-users] speed up backups
2010-05-26 03:51:54
Hello,
Have you tried the rsync method, it should be way faster than tar.
Cheers,
Pedro
On Wednesday 26 May 2010 08:27:04 Ralf Gross wrote:
> Hi,
>
> I'm using BackupPC without major problems since a few years now. Our
> main fileserver has now reached 3,3 TB and it takes 2 days (18 MB/s)
> to do a full backup with tar method.
>
> I'd like to find out if there is something I can do to speed up the full
> backups without changing the hardware.
>
> The file server and the BackupPC server are connected by Gigabit Ethernet
> links. The BackupPC pool is on a harware raid device (RAID6) with 1 GB
> cache. The setup of the file server is a bit complex with drbd/lvm....
>
> Both server are running debian with xfs as filesystem.
>
> I'd have expected high iowait numbers during backup, either on the file
> server or the BackupPC side. But the numbers look ok, not too much iowait.
>
>
> BackupPC server:
>
> 04:35:01 CPU %user %nice %system %iowait %steal %idle
> 04:45:01 all 37.97 0.00 7.29 10.86 0.00 43.88
> 04:55:01 all 35.87 0.00 5.32 9.11 0.00 49.70
> 05:05:01 all 34.69 0.00 4.96 9.91 0.00 50.44
> 05:15:02 all 41.40 0.00 5.88 6.22 0.00 46.50
> 05:25:01 all 44.81 0.00 6.14 4.18 0.00 44.87
> 05:35:01 all 39.34 0.00 5.41 8.41 0.00 46.84
> 05:45:02 all 44.90 0.00 6.84 3.59 0.00 44.68
> 05:55:02 all 32.12 0.00 5.87 5.90 0.00 56.12
> 06:05:01 all 34.23 0.00 6.32 6.28 0.00 53.17
> 06:15:02 all 30.66 0.00 6.01 7.31 0.00 56.01
> 06:25:01 all 18.76 0.00 3.69 7.38 0.00 70.17
> 06:35:03 all 22.08 0.00 5.27 6.61 0.00 66.04
> 06:45:01 all 39.50 0.00 11.54 0.37 0.00 48.59
> 06:55:01 all 37.16 0.00 9.91 2.18 0.00 50.75
> 07:05:02 all 24.52 0.00 4.99 8.99 0.00 61.50
> 07:15:01 all 11.46 0.00 2.65 13.12 0.00 72.77
> 07:25:02 all 11.65 0.00 3.16 11.16 0.00 74.03
> 07:35:01 all 25.32 0.00 5.48 7.22 0.00 61.97
> 07:45:01 all 26.68 0.00 6.71 6.99 0.00 59.62
> 07:55:02 all 29.74 0.00 5.80 4.08 0.00 60.38
> 08:05:01 all 42.30 0.00 6.34 3.49 0.00 47.87
> 08:15:01 all 18.21 0.00 4.26 21.14 0.00 56.39
> 08:25:01 all 25.73 0.00 5.32 20.17 0.00 48.78
> 08:35:01 all 34.94 0.00 6.42 5.61 0.00 53.03
> 08:45:02 all 26.25 0.00 5.00 10.71 0.00 58.04
> 08:55:01 all 48.16 0.00 8.18 0.53 0.00 43.14
> 09:05:01 all 44.54 0.00 7.10 2.25 0.00 46.11
> Average: all 29.70 0.00 5.48 8.36 0.00 56.46
>
> file server:
>
> 06:25:01 CPU %user %nice %system %iowait %steal %idle
> 06:35:01 all 5.91 0.10 7.25 5.71 0.00 81.03
> 06:45:01 all 6.08 0.00 9.49 4.91 0.00 79.52
> 06:55:01 all 5.71 0.00 8.72 3.91 0.00 81.65
> 07:05:01 all 5.67 0.00 6.89 4.16 0.00 83.28
> 07:15:01 all 5.59 0.00 5.72 5.89 0.00 82.79
> 07:25:01 all 5.24 0.00 9.86 6.27 0.00 78.62
> 07:35:01 all 5.91 0.00 13.82 5.10 0.00 75.17
> 07:45:01 all 5.10 0.00 7.54 4.82 0.00 82.55
> 07:55:01 all 4.35 0.00 6.69 3.53 0.00 85.43
> 08:05:01 all 1.77 0.00 3.57 2.17 0.00 92.49
> 08:15:01 all 1.81 0.00 2.44 3.73 0.00 92.02
> 08:25:01 all 2.10 0.00 4.75 3.08 0.00 90.07
> 08:35:01 all 2.10 0.00 6.37 3.60 0.00 87.92
> 08:45:01 all 2.31 0.00 5.09 3.46 0.00 89.15
> 08:55:01 all 2.05 0.00 4.06 1.34 0.00 92.56
> 09:05:01 all 2.00 0.00 3.63 2.31 0.00 92.06
> Average: all 2.50 0.00 3.84 2.91 0.00 90.75
>
>
> After all I've read, switching to rsync instead of tar doesn't seem to be a
> better choice.
>
> Disk I/O on the file server doesn't seem to be the bottleneck either. I can
> boost the disk I/O during backup with other tools (dd, cat, bonnie++) to more
> than 50 MB/s.
>
> Any ideas if I can tune my BackupPC settings to speed things up?
>
> Ralf
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> BackupPC-users mailing list
> BackupPC-users AT lists.sourceforge DOT net
> List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki: http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
>
--
----------------------------------------------------------------------------------------------------------
Pedro M. S. Oliveira
IT Consultant
Email: pmsoliveira AT gmail DOT com
URL: http://www.linux-geex.com
Cellular: +351 96 5867227
----------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
|
|
|
