We have seen under windows that directories containing more than 10,000
files tend to slow the backup process down - on unix it was a tad more -
more than 100,000.
This is a operating system "limitation" and has is the way that the Os walks
the filesystem - nothing to do with networker (I have seen this on Laptop,
NRM, DiskXtender etc etc) ...
We have had a lot of improvement by getting clients to try and limit files
in a directory to less than 10,000 (i.e. create sensible subdirectories).
Regards,
Riaan
-----Original Message-----
From: Lubos Bohm [mailto:lubos.bohm AT OSKARMOBIL DOT CZ]
Sent: Thursday, July 24, 2003 2:48 PM
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Subject: [Networker] Large directories slow acces using save -I
input_filename
Hi All,
trying to backup aprox. 200.000 files listed in "input_filename" from large
directory containing aprox. 960.000 files takes IMHO too long. Well, it
takes hours before the tape is even loaded and first bytes are written..
We use:
1/ save -s lgt-blan -q -I /var/opt/Storage/Apr.in -b"BGW"
on Solaris 8 with VXFS
2/ Networker 6.1.1 build 238 on both sides
I suppose the problem is inside the "save" - it can not search through the
large directory.
Folks, what are your experiences? Could it be a bug solved in new version?
Or should I better take care about pre-processing these files like moving
them in a directory tree (like for example Squid does)? Or some advanced
VxFS tuning?
Thanks a lot, Lubos
--
Note: To sign off this list, send a "signoff networker" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
--
Note: To sign off this list, send a "signoff networker" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
|