Re: [BackupPC-users] Backuppc performance

Hi

On Mon, 26 Dec 2016 22:52:39 +0100, Pince Sans Rire <sargastic AT gmail DOT com>
wrote:
> Everything works okay, *except* the backup thoughput. Here is a copy of
> the "File size/count reuse summary" for one of the backed-up computers :

        You do not say what type of backup are you using, usually
rsync+ssh is the more flexible one, but is also uses ssh that will
default for security, not speed. Rsync+rsyncd will be faster, but
without any encryptation.

        You can request some special ciphers to try to workaround this,
in the past, arcfour was the fastest (but more insecure), today many 
CPUs have AES extensions and ciphers that use that will usually be
faster:

https://blog.famzah.net/2015/06/26/openssh-ciphers-performance-benchmark-update-2015/

        Test your cipher performance (both on server and client) and
choose the best one for your system. If arcfour, you will have to enable
it on the server, by adding it to the end of the list. Then you enforce
the best cipher in the backuppc rsync command with the -e "ssh -c
(cipher)" or by adding the cipher to the backuppc .ssh/config

        Of course, CPU, IO and network may also be a problem, so
check each one to see if you do not have any bottleneck on any of
then. Use htop, vmstat, iostat, bmon for that

> This seems really, really low (but even the 1.1 MB/sec maximum is
> exceedingly low). I have full backups (around 400 Gb to copy) that time
> out after 15 or 20 hours of backup. This is bothersome.

        Another reason may be a very high number of files/directories, 
that may be forcing rsync to check many entries. Even if you exclude
most of it, many directories may still need to be checked to find 
other files not excluded.

        If the backup size is really big and slow to check, you can
break the backup in several ones, so each one be smaller and faster.
I do this on a server with several million of files and directories
and took almost one day to finish. After breaking it in about 10
blocks, each one takes about 2hours... but resuming and incremental
backups are much faster. You may need to tune a little the blackout
hours so you do not have many parallel backups on the same machine.

good luck
Higuita
-- 
Naturally the common people don't want war... but after all it is the
leaders of a country who determine the policy, and it is always a 
simple matter to drag the people along, whether it is a democracy, or
a fascist dictatorship, or a parliament, or a communist dictatorship.
Voice or no voice, the people can always be brought to the bidding of
the leaders. That is easy. All you have to do is tell them they are 
being attacked, and denounce the pacifists for lack of patriotism and
exposing the country to danger.  It works the same in every country.
           -- Hermann Goering, Nazi and war criminal, 1883-1946

pgpwmCG1FFHIV.pgp
Description: OpenPGP digital signature

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot

_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/