BackupPC-users

Re: [BackupPC-users] Backing up many, many files over a medium speed link - kills CPU and fails halfway. Any help?

2009-11-24 20:08:51
Subject: Re: [BackupPC-users] Backing up many, many files over a medium speed link - kills CPU and fails halfway. Any help?
From: Chris Bennett <chris AT ceegeebee DOT com>
To: GB <pseudo AT gmail DOT com>
Date: Wed, 25 Nov 2009 11:35:47 +1030
Hi there,

> Can anybody suggest some intelligent solutions for doing this? How can I
> speed up rsync / set it to use less CPU? I searched for some solutions, but
> everyone talks about limiting rsync _transfer_ speed, which is hardly the
> issue here - I'm backing up over a cable modem, heh.
> 
> Any help appreciated. Thanks!

I havn't thought too much on your actual problem, but just in case you
hadn't considered it, can you change the backup source to suit your
backup system?  e.g. does your 370,000 files represent
days/weeks/years of information?  Can this be archived by month etc,
so the file count is massively reduced?

I see customers with web developers with little sysadmin skills and
they produce the most enormous trees of never-changing files spanning
days/months/years and backuppc needs to check these files everytime,
which hurts the overrall throughput.

Similarly, have you observed excessive swap in/out on your backuppc
server?  If you strace the rsync process on the remote system, does it
appear to be very 'bursty' in I/O activity, and overall throughput
just looks 'wrong'?

Regards,

Chris Bennett
cgb

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/