-------- Original-Nachricht --------
> Datum: Wed, 3 Mar 2010 09:16:18 -0500
> Von: "STANLEY R. HORWITZ" <stan AT temple DOT edu>
> An: EMC NetWorker discussion <NETWORKER AT LISTSERV.TEMPLE DOT EDU>, Ronny 
> Egner <RonnyEgner AT GMX DOT DE>
> Betreff: Re: [Networker] Splitting up large directory save jobs into smaller 
> pieces (aka. "parallelizing")

> 
> On 03 3, 2010, at 9:12 AM, Ronny Egner wrote:
> 
> > 
> > I am aware this problem can be solved by using NDMP and some kind of
> block based backup approach rather than a file based one. But i´d like to
> have a script a hand which automatically splits up large save jobs into
> smaller ones (a guess i am not the only one). 
> > 
> > Feedbacks? Comments? Any other ideas? 
> 
> You have a rough road to travel in either approach. Much better is for
> your clients to organize their file systems in an optimal way that facilitates
> effective backups and overall use of the data on their servers. Just
> piling files and directories onto a file system without concern for how the 
> data
> will be protected is a poor strategy and scripting one's way around that
> deficiency isn't a good idea in my humble opinion, regardless of which
> backup solution is in use.

Hi Stanley,

thanks for the fast response. I completely agree with you. However reality 
often looks different. Often storage is used without any structure. Or it
is used sub-optimally because the developer decided "it is the best to do 
so". Rewriting takes too much time... i guess you know that from 
experience.

For instance a custom written revision tracking system which creates a
separate directory containing the compressed file FOR EACH REVISION OF THE 
FILE.
So i am facing a directory with approx half a million sub-directories 
(growing in number!) each containing one single file. Running 
non-parallelized  takes ages (no wonder why).


Ronny
-- 
Ronny Egner
RonnyEgner AT gmx DOT de

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the body of the email. Please write to 
networker-request AT listserv.temple DOT edu if you have any problems with this 
list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER