BackupPC-users

Re: [BackupPC-users] (somewhat solved) Backing up many small files

2013-02-06 05:02:59
Subject: Re: [BackupPC-users] (somewhat solved) Backing up many small files
From: "Sorin Srbu" <sorin.srbu AT orgfarm.uu DOT se>
To: "'General list for user discussion, questions and support'" <backuppc-users AT lists.sourceforge DOT net>
Date: Wed, 6 Feb 2013 11:01:51 +0100
> -----Original Message-----
> From: Tyler J. Wagner [mailto:tyler AT tolaris DOT com]
> Sent: Wednesday, February 06, 2013 10:47 AM
> To: General list for user discussion, questions and support
> Cc: Sorin Srbu
> Subject: Re: [BackupPC-users] (somewhat solved) Backing up many small
> files
> 
> > I realized that afterwards. I'll be looking for some kind of random test
> > instead.
> 
> No, this is what you want:
> 
> /usr/bin/time -v find /home -mtime 1 >/dev/null
> 
> This will generate a list of all files in /home, checking each of their
> modification times, and throw all the output away. This performs a
> total
> directory traversal where each file's inode is checked. This is almost
> certainly the limiting factor of your rsync.
> 
> The above spits out a lot of output. You are interested in "User time"
> (CPU
> time in userspace) and "Elapsed (wall clock) time". Example from my PC:
> 
> tyler@baal:~$ /usr/bin/time -v find /home -mtime 1 >/dev/null
>       Command being timed: "find /home -mtime 1"
>       User time (seconds): 0.14
>       System time (seconds): 0.96
>       Percent of CPU this job got: 15%
>       Elapsed (wall clock) time (h:mm:ss or m:ss): 0:07.07
>       Average shared text size (kbytes): 0
>       Average unshared data size (kbytes): 0
>       Average stack size (kbytes): 0
>       Average total size (kbytes): 0
>       Maximum resident set size (kbytes): 2320
>       Average resident set size (kbytes): 0
>       Major (requiring I/O) page faults: 0
>       Minor (reclaiming a frame) page faults: 1512
>       Voluntary context switches: 2398
>       Involuntary context switches: 196
>       Swaps: 0
>       File system inputs: 31000
>       File system outputs: 0
>       Socket messages sent: 0
>       Socket messages received: 0
>       Signals delivered: 0
>       Page size (bytes): 4096
>       Exit status: 0

Thanks! Was just about to start some searching. 8-)

Running the suggested test on both client and server as I write this.


> Directory and inode data is cached by the kernel so subsequent runs are
> faster. You can tune these with sysctl:
> 
> vm.swappiness
> vm.vfs_cache_pressure
> 
> More information (shameless use of my blog):
> 
> http://www.tolaris.com/2008/09/28/making-the-gui-faster-in-ubuntu-linux/

Will check this as soon as I get some numbers back from the tests. Might
take some time...

-- 
/Sorin

Attachment: smime.p7s
Description: S/MIME cryptographic signature

------------------------------------------------------------------------------
Free Next-Gen Firewall Hardware Offer
Buy your Sophos next-gen firewall before the end March 2013 
and get the hardware for free! Learn more.
http://p.sf.net/sfu/sophos-d2d-feb
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
<Prev in Thread] Current Thread [Next in Thread>