Re: [BackupPC-users] Performance reference (linux --(rsync)-> linux)

Dear all,

For reference on the matter I was trying to resolve or improve, I have increased RAM from 2Gb to 4Gb (the max for that machine) and backups reduced by 50% in completion time.

I had wrongly assessed in the past that it was taking 10 days to complete a full backup, when it actually took around 7 days.

This has now been brought down to 3.5 days.

Incremental is now at 9 hours which is reasonable, albeit still on the slow side.

We will try to improve things further based on all your much appreciated information.

Thanks!

--------------------------
Cassiano Surek
cass AT surek.co DOT uk

On 7 Nov 2012, at 17:25, Les Mikesell wrote:

On Wed, Nov 7, 2012 at 10:42 AM, Timothy J Massey <tmassey AT obscorp DOT com> wrote:

Cassiano Surek <cass AT surek.co DOT uk> wrote on 11/06/2012 05:03:44 AM:

Of course, how could I have missed that! I did find it now, thanks Michał.

Last full backup (of 100 odd Gb) took slightly north of 10 days to
complete. Incremental, just over 5 days.

I did not see if you mentioned how *many* files you had. Are we talking 100,000 files or 10 Million files? That will make a *big* difference in performance.

This will be especially true if the full directory listing sent before
comparison starts fills RAM and pushes into swap. A couple of other
performance-killers that haven't been mentioned yet:

Running too many backups concurrently - on some systems 2 might be too many.

Having large files that change frequently. This forces the server to
uncompress the base copy and merge changes which will thresh the disk
since they are on the same drive.   This may be unavoidable, but it
might be worth excluding things like log files you don't care about,
database files that won't be consistent anyway, etc..

--
  Les Mikesell
    lesmikesell AT gmail DOT com

------------------------------------------------------------------------------
LogMeIn Central: Instant, anywhere, Remote PC access and management.
Stay in control, update software, and manage PCs from one command center
Diagnose problems and improve visibility into emerging IT issues
Automate, monitor and manage. Do more in less time with Central
http://p.sf.net/sfu/logmein12331_d2d
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

------------------------------------------------------------------------------
Monitor your physical, virtual and cloud infrastructure from a single
web console. Get in-depth insight into apps, servers, databases, vmware,
SAP, cloud infrastructure, etc. Download 30-day Free Trial.
Pricing starts from $795 for 25 servers or applications!
http://p.sf.net/sfu/zoho_dev2dev_nov

_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/