BackupPC-users

[BackupPC-users] rsync building file list takes forever

2014-03-01 11:22:31
Subject: [BackupPC-users] rsync building file list takes forever
From: "Dr. Boris Neubert" <omega AT online DOT de>
To: BackupPC-users AT lists.sourceforge DOT net
Date: Sat, 01 Mar 2014 17:20:56 +0100
Dear BackupPC users,

I have an issue with rsyncd seeming to take forever building the file
list on a otherwise powerful client. I hope you have got a suggestion
how to address the issue. I found reports of similar issues in my web
searches and mailing list archive search but no hint how to solve my
problem.


I recently switched my environment to:

BackupPC server:
    BackupPC 4.0.0.alpha3 on CentOS Linux 6.5 64bit, 4 GB memory, AMD
Athlon LE-1640 1 core processor

Client #1: NAS:
    FreeNAS 9.2.1 on FreeBSD 9.2, 16 GB memory, Intel Xeon E3-1230v3 4
core hyperthreading processor

Client #2: Virtual Machine Server:
    CentOS Linux 6.5 64bit, 16 GB memory, Intel Xeon E3-1230v3 4 core
hyperthreading processor

The bulk of data is on the NAS, a total of about 3 TB, partitioned into
rsync shares, the largest of which is the "video" share with 650 GB in
4,000 video files. Full backups are done in reasonable times (below 2
hours) and incremental backups are incredibly fast (< 1 minute) if no
files have changed.

On the other hand, rsync 3.0.6 on the Virtual Machine Server takes hours
and hours to build the file list - actually I never waited for this to
finish. We talk about 1,800 files, most of them small log files and the
bulk of the 578 GB of data in maybe 50 large virtual disk files and a
handful of ISO image dumps of DVDs. The underlying file system for
backup is a snapshot of a logical volume (LVM2). File system type is xfs.

This renders it impossible to backup the Virtual Machine Server.

The predecessor of the Virtual Machine Server was OpenSUSE 12.3 with XFS
and no LVM on a consumer type PC. Backups where done within a few hours.

While I am writing this I can observe the output of strace -p <PID of
rsyncd on client #2> scrolling through. There are some intermittent
calls to select that timeout. The hosts XferLOG is still empty.

I tried running rsync on the BackupPC server with and without the
--checksum option but did not see any difference.

Thanks for your patience to read so far.

Kind regards,
Boris






------------------------------------------------------------------------------
Flow-based real-time traffic analytics software. Cisco certified tool.
Monitor traffic, SLAs, QoS, Medianet, WAAS etc. with NetFlow Analyzer
Customize your own dashboards, set traffic alerts and generate reports.
Network behavioral analysis & security monitoring. All-in-one tool.
http://pubads.g.doubleclick.net/gampad/clk?id=126839071&iu=/4140/ostg.clktrk
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

<Prev in Thread] Current Thread [Next in Thread>