In regard to: [Networker] Splitting up large directory save jobs into...:
One customer asked me if it is possible to speed up saves of large
directories with many sub-dirs (or files). Unfortunately Networker itself
cannot automatically parallelize saving of directories. Thats quite odd.
The ability to automatically parallelize different parts of one filesystem
has been on the list of desired features for a long time. That feature
was included in a previous release of the product (I think it was 7.3,
but I would have to check my notes to be certain) as a hidden,
experimental feature. It was subsequently either removed or completely
disabled.
So i wrote simple script bash script in order to split up large
directories in smaller (and parallel) save jobs.
No script that's going to do it robustly is going to remain simple, I
think. :-|
Most of the discussion on the list in the past about this problem has
centered around the "split client" trick, which does not automatically
split things (the administrator has to set up how a volume is split) but
is "safe", if done correctly, because one of the two (or more) clients
uses the All saveset.
Tim
--
Tim Mooney Tim.Mooney AT ndsu DOT
edu
Enterprise Computing & Infrastructure 701-231-1076 (Voice)
Room 242-J6, IACC Building 701-231-8541 (Fax)
North Dakota State University, Fargo, ND 58105-5164
To sign off this list, send email to listserv AT listserv.temple DOT edu and type
"signoff networker" in the body of the email. Please write to networker-request
AT listserv.temple DOT edu if you have any problems with this list. You can access the
archives at http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
|