BackupPC-users

Re: [BackupPC-users] (somewhat solved) Backing up many small files

2013-02-05 10:40:23
Subject: Re: [BackupPC-users] (somewhat solved) Backing up many small files
From: "Sorin Srbu" <sorin.srbu AT orgfarm.uu DOT se>
To: "'General list for user discussion, questions and support'" <backuppc-users AT lists.sourceforge DOT net>
Date: Tue, 5 Feb 2013 16:38:25 +0100
> -----Original Message-----
> From: Sorin Srbu [mailto:sorin.srbu AT orgfarm.uu DOT se]
> Sent: Tuesday, February 05, 2013 12:40 PM
> To: 'General list for user discussion, questions and support'
> Subject: Re: [BackupPC-users] Backing up many small files
> 
> > -----Original Message-----
> > From: Markus [mailto:universe AT truemetal DOT org]
> > Sent: Tuesday, February 05, 2013 11:58 AM
> > To: General list for user discussion, questions and support
> > Subject: Re: [BackupPC-users] Backing up many small files
> >
> > I posted basically your same question a few months ago and with the
> > help
> > of the friendly people on this list it turned out that I just have
> > simply way too many files on the client, about 25 million actually. Do
> > a "find / | wc -l" to see how many files you got! My find run alone took
8
> > hours if I remember correctly. So how is rsync supposed to do it any
> > faster :)  You can check the thread here, people share some really good
> > infos: http://sourceforge.net/mailarchive/message.php?msg_id=30104262
> 
> Oh, yeah. I remember that one now, I followed it rather closely for a
> while. Thanks for the reminder!
> 
> Doing the suggested find-operation. Will be back in a while with more
> info.

Only about 2 milion files.
user@BPC-client ~/ [0]# find /home | wc -l
1883781

And the test incremental backup took 118.5 min's for 61 MB.

Somewhat better I guess after the noa* tweaks. I'm inclined to believe that
the bootle-neck is, as Markus points out, that the client maybe isn't to
top-notch and the sheer number of files to be checked.

I'll leave this issue as somewhat solved and maybe wait for BPC to implement
a more modern rsync-strategi as mention in the archived thread above.

Thanks everybody!

-- 
/Sorin


Attachment: smime.p7s
Description: S/MIME cryptographic signature

------------------------------------------------------------------------------
Free Next-Gen Firewall Hardware Offer
Buy your Sophos next-gen firewall before the end March 2013 
and get the hardware for free! Learn more.
http://p.sf.net/sfu/sophos-d2d-feb
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
<Prev in Thread] Current Thread [Next in Thread>