BackupPC-users

Re: [BackupPC-users] rsyncd full backup

2013-10-31 15:14:12
Subject: Re: [BackupPC-users] rsyncd full backup
From: Timothy J Massey <tmassey AT obscorp DOT com>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Thu, 31 Oct 2013 15:06:36 -0400
Sharuzzaman Ahmat Raslan <sharuzzaman AT gmail DOT com> wrote on 10/31/2013 02:38:01 PM:

> Hi Timothy,
>

> I got the number by observing the output of iotop while file
> transfer is running. Also, on BackupPC host summary page, average
> transfer rate for full backup is also around 3MB/s

> It could be a network bottleneck also, as the customer is using
> 100Mbps switch with around 80 PC, not including network printer and
> servers. Inclusive should be around 100 network devices.


For file transfers, 100Mb/s is good for 7MB/s transfer rate.  Assuming a good quality switch (which is a *big* assumption), the number of computers shouldn't matter.


But I would think strongly about buying a good quality Gigabit switch (I recommend the HP V1910-24G) as your "backbone":  Plug all of your servers (including the BackupPC server) into it, as well as each of your 100Mb/s switches (even better if they have Gb uplink ports!).  That would eliminate the network as a bottleneck and only costs $300.  And improve network performance across the board, though your users may not notice it if they only work with small files.

> Any idea how to properly troubleshoot network bottleneck? My skill
> is a little bit lacking on that area.

Sure:  Time the copying of files from one machine to another.  Assuming the source and destination hard drives are faster than 7MB/s (and they very well *better* be!), then you'll saturate a 100Mb network no problem.

For a more scientific approach, check out iperf.

I'd be *much* more worried about checking out your *disk* performance.  You can do tests in exactly the same way:  copy files to and from the disk and see what happens.  Here are some very simple examples:

sync; time dd if=/dev/zero of=test.fil bs=1M count=1024; sync; sync; sync;
sync; time dd if=test.fil of=/dev/null bs=1M

The first line times the writing of a 1GB file named test.fil.  The second one times the reading of the same 1GB file.  Divide 1024 by the number of seconds it takes and that will give you the MB/s that you transferred.  (The sync command is needed for accurate timing;  the three sync commands is kind of an old UNIX graybeard joke:  http://unix.stackexchange.com/questions/5260/is-there-truth-to-the-philosophy-that-you-should-sync-sync-sync-sync )

If you want more scientific disk performance information, check out iozone or iometer.

Remember:  always profile before you optimize.  ( http://www.phatcode.net/res/224/files/html/ch37/37-02.html )

Tim Massey

 
Out of the Box Solutions, Inc.
Creative IT Solutions Made Simple!

http://www.OutOfTheBoxSolutions.com
tmassey AT obscorp DOT com
      22108 Harper Ave.
St. Clair Shores, MI 48080
Office: (800)750-4OBS (4627)
Cell: (586)945-8796

------------------------------------------------------------------------------
Android is increasing in popularity, but the open development platform that
developers love is also attractive to malware creators. Download this white
paper to learn more about secure code signing practices that can help keep
Android apps secure.
http://pubads.g.doubleclick.net/gampad/clk?id=65839951&iu=/4140/ostg.clktrk
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
<Prev in Thread] Current Thread [Next in Thread>