Re: [BackupPC-users] (somewhat solved) Backing up many small files
2013-02-06 05:02:59
> -----Original Message-----
> From: Tyler J. Wagner [mailto:tyler AT tolaris DOT com]
> Sent: Wednesday, February 06, 2013 10:47 AM
> To: General list for user discussion, questions and support
> Cc: Sorin Srbu
> Subject: Re: [BackupPC-users] (somewhat solved) Backing up many small
> files
>
> > I realized that afterwards. I'll be looking for some kind of random test
> > instead.
>
> No, this is what you want:
>
> /usr/bin/time -v find /home -mtime 1 >/dev/null
>
> This will generate a list of all files in /home, checking each of their
> modification times, and throw all the output away. This performs a
> total
> directory traversal where each file's inode is checked. This is almost
> certainly the limiting factor of your rsync.
>
> The above spits out a lot of output. You are interested in "User time"
> (CPU
> time in userspace) and "Elapsed (wall clock) time". Example from my PC:
>
> tyler@baal:~$ /usr/bin/time -v find /home -mtime 1 >/dev/null
> Command being timed: "find /home -mtime 1"
> User time (seconds): 0.14
> System time (seconds): 0.96
> Percent of CPU this job got: 15%
> Elapsed (wall clock) time (h:mm:ss or m:ss): 0:07.07
> Average shared text size (kbytes): 0
> Average unshared data size (kbytes): 0
> Average stack size (kbytes): 0
> Average total size (kbytes): 0
> Maximum resident set size (kbytes): 2320
> Average resident set size (kbytes): 0
> Major (requiring I/O) page faults: 0
> Minor (reclaiming a frame) page faults: 1512
> Voluntary context switches: 2398
> Involuntary context switches: 196
> Swaps: 0
> File system inputs: 31000
> File system outputs: 0
> Socket messages sent: 0
> Socket messages received: 0
> Signals delivered: 0
> Page size (bytes): 4096
> Exit status: 0
Thanks! Was just about to start some searching. 8-)
Running the suggested test on both client and server as I write this.
> Directory and inode data is cached by the kernel so subsequent runs are
> faster. You can tune these with sysctl:
>
> vm.swappiness
> vm.vfs_cache_pressure
>
> More information (shameless use of my blog):
>
> http://www.tolaris.com/2008/09/28/making-the-gui-faster-in-ubuntu-linux/
Will check this as soon as I get some numbers back from the tests. Might
take some time...
--
/Sorin
smime.p7s
Description: S/MIME cryptographic signature
------------------------------------------------------------------------------
Free Next-Gen Firewall Hardware Offer
Buy your Sophos next-gen firewall before the end March 2013
and get the hardware for free! Learn more.
http://p.sf.net/sfu/sophos-d2d-feb _______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
|
<Prev in Thread] |
Current Thread |
[Next in Thread>
|
- Re: [BackupPC-users] Backing up many small files, (continued)
- Message not available
- Re: [BackupPC-users] (somewhat solved) Backing up many small files, Les Mikesell
- [BackupPC-users] Finding hosts on a home LAN, Magnus Larsson
- Re: [BackupPC-users] Finding hosts on a home LAN, Les Mikesell
- Re: [BackupPC-users] Finding hosts on a home LAN, Tyler J. Wagner
- Re: [BackupPC-users] (somewhat solved) Backing up many small files, Sorin Srbu
- Re: [BackupPC-users] (somewhat solved) Backing up many small files, Tyler J. Wagner
- Re: [BackupPC-users] (somewhat solved) Backing up many small files,
Sorin Srbu <=
- Re: [BackupPC-users] (somewhat solved) Backing up many small files, Sorin Srbu
- Re: [BackupPC-users] (somewhat solved) Backing up many small files, Tyler J. Wagner
- Re: [BackupPC-users] (somewhat solved) Backing up many small files, Les Mikesell
- Re: [BackupPC-users] (somewhat solved) Backing up many small files, Sorin Srbu
- Message not available
- Re: [BackupPC-users] (somewhat solved) Backing up many small files, Les Mikesell
- Re: [BackupPC-users] (somewhat solved) Backing up many small files, Sorin Srbu
Re: [BackupPC-users] Backing up many small files, Les Mikesell
|
|
|