On Tue, Dec 15, 2009 at 06:27:41AM -0600, Carl Wilhelm Soderstrom
wrote:
> On 12/14 04:25 , Robin Lee Powell wrote:
> > Not with large trees it isn't.  I have 3.5 million files, and
> > more than 300GiB of data, in one file system.  The last
> > incremental took *twenty one hours*.  I have another backup
> > that's 4.5 million files, also more than 300 GiB of data, also
> > in one file system.  The full took 20 hours; it hasn't succeeded
> > at an incremental yet.  That's over full 100BaseT, if not better
> > (I'm not the networking person).
> 
> Pardon me if this has been mentioned already and I missed it, but
> I presume you've already gone over both ends of the connection and
> checked for possible optimizations?

Just to give a sense of scale here:

# date ; find /pictures -xdev -type f -printf "%h\n" >/tmp/dirs ; date
Tue Dec 15 12:50:57 PST 2009
Tue Dec 15 17:26:44 PST 2009

(something I ran to try to figure out how to partition the tree)

The other one isn't even close to finishing, as far as I can tell.
In the face of it taking nigh-on 5 hours just to *walk the tree*,
from the local host, I haven't been focusing on little things like
ssh encryption choices too much.  :)

As a general comment to everybody: I *am* reading what you're
saying, I do appreciate the help, and I'm going to try out some of
the things you suggested. Thank you.

-Robin

-- 
They say:  "The first AIs will be built by the military as weapons."
And I'm  thinking:  "Does it even occur to you to try for something
other  than  the default  outcome?"  See http://shrunklink.com/cdiz
http://www.digitalkingdom.org/~rlpowell/ *** http://www.lojban.org/

------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community
Take advantage of Verizon's best-in-class app development support
A streamlined, 14 day to market process makes app distribution fast and easy
Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev 
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/