Re: [BackupPC-users] Rsync to Windows client very slow (Was: Re:How to BackupFilesExclude to exclude locked files on Windows XP)
2008-05-25 15:44:05
Hi again,
>It would be nice to
be able to use WindowsXP+'s index as a reference or have the rsyncd on the
windows machine maintain all the checksum information but I know of no way and
no plans to do this.
Another option would be the archive bit, wouldn't
it?
Greetings,
Hendrik
rsync backups are quite slow. over slow links this is more
than made up for by the decrease in network bandwidth used but on a high speed
lan it is very slow. I am on gigabit from my desktop to my backuppc
server but rarely exceed 6-8MB/s during a backup.
That makes
sense some of the time as rsync will only be transfering checksum data but
when there is a large file being transfered it is still quite slow.
install iftop and watch a backup that has a new large file that is not in an
exsisting backup and you will not see the transfer speed at anywhere near the
LAN speed or hard disk speed because of I/O bottlenecks.
It would be
nice to be able to use WindowsXP+'s index as a reference or have the rsyncd on
the windows machine maintain all the checksum information but I know of no way
and no plans to do this.
I/O is an obvious slow point here but rsync
doesn't seem to be aware of I/O bottlenecks. it seems to scan files at
random rather than trying to scan files in the on-disk-order on the client
side and then cache disk writes on the server side to overcome the I/O
bottleneck. Another, possibly the primary factor, in the I/O bottleneck
is the way almost all modern filesystems handle small file creation, which
includes all files smaller than the cluster size but also directories,
softlinks, and hardlinks.
The arguement of storing files in a database
has come up as a database is better able to overcome the I/O deficites of the
disk storage by some intelegent caching but that would require more complex
code. Alternatively using a fuse-based SQL filesystem would help
overcome the I/O issue by taking advantage of MySQL's caching but fuse modules
are notoriously innefficient and the overhead would negate the
benefit.
One functional solution is to use a filesystem that has the
caching benefits without any extra complexity. ZFS on BSD and Solaris
fits the bill and performs very very well with backuppc. Unfortunately,
none of the BSDs, Solaris, or openSolaris include backuppc and filling
dependancies is just beyond many users as a large number of backuppc
users/admins are all about Linux. In fact, ZFS can have cache disks in
the storage pools and pre-write data sequentially from the RAM cache for
re-organization later, making ZFS very effective at hiding the I/O
bottleneck.
On Wed, May 21, 2008 at 4:19 AM, Kurt Jasper < kurt.jasper AT googlemail DOT com>
wrote:
Ok,
thanks
for (all) the information, now I understand the concept
of Rsync/SambaShareName, BackupFilesExlude and BackupFilesOnly. You
really helped me out, Thanks :-)
But unfortunately I've discovered
another problem:
The backups of my windows xp clients take forever
(nearly), while the backup of the server (where backuppc is runnning)
works fine. I've done some testing and figured out that it is NOT an
network issue, since I get the full speed when executing rsync from the
windows xp client.
I have transferred the cygwin directory of one of
the clients (approx. 300 MB):
On Windows-Client: rsync -av -h
--progress /cygdrive/c/cygwin backuppc@server:~/ => works
fine!
On Server: rsync -av -h --progress
root@winxpclient:/cygdrive/c/cygwin ~/ => takes very, verrrry
long.
Browsing the web, I've found out that it may have something to
do with the Cygwin Enviroment Variables. I'm currently using the
suggested settings: CYGWIN = ntsec tty.
Question: 1) So what
may be the problem? What is the best way to back up windows xp clients? I
don't understand that running rsync from CLIENT-to-SERVER runs fine, but
running from SERVER-to-CLIENT is really slow. 2) Is is better to use
Samba to backup windows clients? Some clients are running XP
(Home), so that I can not set custom file permissions to allow access
only the backuppc-user not to other clients in the network. 3) Will
rsyncd instead of rsync bring better performance?
Kind
regards
- Kurt
PS: To give at least something back to this
community, I thought about writing down my experiences (from a beginners
point of view), I hope this is the right place: http://backuppc.wiki.sourceforge.net/CommonConfigurationIssues If
not feel free to flame me :-0
Bowie Bailey wrote:
> Your
RsyncShareName is '/', but your are specifying exclusions for the >
'c' share. Try this instead: > >
$Conf{BackupFilesExclude} = { > '/' =>
['/Dokumente und Einstellungen/*/NTUSER.DAT', .... >
};
------------------------------------------------------------------------- This
SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R)
Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ BackupPC-users
mailing list BackupPC-users AT lists.sourceforge DOT net List:
https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:
http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
|
<Prev in Thread] |
Current Thread |
[Next in Thread>
|
- Re: [BackupPC-users] How to BackupFilesExclude to exclude locked files on Windows XP, Bowie Bailey
- [BackupPC-users] Rsync to Windows client very slow (Was: Re: How to BackupFilesExclude to exclude locked files on Windows XP), Kurt Jasper
- Re: [BackupPC-users] Rsync to Windows client very slow (Was: Re: How to BackupFilesExclude to exclude locked files on Windows XP), dan
- Re: [BackupPC-users] Rsync to Windows client very slow (Was: Re:How to BackupFilesExclude to exclude locked files on Windows XP),
Hendrik Friedel <=
- Re: [BackupPC-users] Rsync to Windows client very slow (Was: Re:How to BackupFilesExclude to exclude locked files on Windows XP), Kurt Jasper
- Re: [BackupPC-users] Rsync to Windows client very slow (Was: Re:How to BackupFilesExclude to exclude locked files on Windows XP), dan
- Re: [BackupPC-users] Rsync to Windows client very slow (Was: Re:How to BackupFilesExclude to exclude locked files on Windows XP), Nils Breunese (Lemonbit)
- Re: [BackupPC-users] Rsync to Windows client very slow (Was: Re:How to BackupFilesExclude to exclude locked files on Windows XP), dan
- Re: [BackupPC-users] Rsync to Windows client very slow, Jonathan Dill
- Re: [BackupPC-users] Rsync to Windows client very slow, Kurt Jasper
- Re: [BackupPC-users] Rsync to Windows client very slow, Jonathan Dill
- Re: [BackupPC-users] Rsync to Windows client very slow, Stephen Joyce
- Re: [BackupPC-users] Rsync to Windows client very slow, Hendrik Friedel
- Re: [BackupPC-users] Rsync to Windows client very slow (Was: Re:Howto BackupFilesExclude to exclude locked files on Windows XP), Hendrik Friedel
|
|
|