Hi list,
I'm new here. At first, thank you for BackupPC! :)
I'm trying to backup a new client. The problem is: rsync never starts to
transfer files, not even after 12 hours of waiting. rsync is doing
something, though. More on that below.
The client is a quad core 2.8 GHz CPU, 8 GB RAM and 1.6 TB of many many
small files in a RAID0. CPUs 75-95% idle most of the time, load around
0.3. No swap used.
rsync 3.0.7 on the client, rsync 3.0.6 on the server. Negotiated
protocol 28 (hmm).
BackupPC server is backing up to a 8 TB iSCSI RAID5 drive.
So, when I initiate the full backup manually, and then attach with
strace to the rsync process on the BackupPC server I see activity, but
it's so slow. Roughly it takes about 5 seconds for 10 lines of
read/select/write. On the client I can see via lsof that rsync is going
through the filesystem and the many mini-files. On the client rsync
consumes like 0.x-1% CPU and the "virt" mem size is growing slowly (80M
after 10 minutes, in the beginning 40M; that's what "top" says).
Load on the client goes from 0.3 to about 1.8 a few minutes after I
start the full backup. But CPUs stay at around 70-90% idle.
Last evening I started the backup at 22:00 and in the morning at around
09:00 the customer killed the rsync procress manually because he said
the client was becoming unresponsive. Even at 09:00 there was not yet a
single file transferred.
I'm guessing the client is slowing down after so many hours because
rsync has used up all memory as it is caching the list of files to be
transferred? Unfortunately I didn't monitor yet what the rsync process
looks like after 12 hours, but will do. I'm now starting the rsync
process with "ionice -c 3 nice -n10 /usr/bin/rsync" because the customer
complained, but if the memory usage is the reason for the client to slow
down, this will probably not help.
What I don't understand yet is - why does rsync on the client tell rsync
on the server about the files it is currently going through? I mean, it
is not even transferring them yet! I see these select/read/write outputs
in strace attached to the rsync server process, and as it appears this
means "Hello, I'm the client, and I'm telling you which files I go
through now on the client filesystem". But why does the rsync server
process even need to know about that at this point in time?
I'm guessing it can't be the comparing mechanism of BackupPC, because
there are no files to compare yet! There are 0 backups with 0 files.
I'm backing up about 25 other clients just fine, but this is the only
one with such a large amount of files and data. This is why I suspect
the issue is with the many files/big data. All other clients have 150 GB
maximum, after backuping. These are kinda slow too, but at least they
already start tranferring files after a couple of seconds, minutes or 1,
2 hours at most.
Lots of guessing ...
Any suggestions on what I could do or what could go wrong here?
Thank you very much!
Markus
------------------------------------------------------------------------------
Monitor your physical, virtual and cloud infrastructure from a single
web console. Get in-depth insight into apps, servers, databases, vmware,
SAP, cloud infrastructure, etc. Download 30-day Free Trial.
Pricing starts from $795 for 25 servers or applications!
http://p.sf.net/sfu/zoho_dev2dev_nov
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
|
