Interesting solution :o)
I have checked out what effect the change journal on w2k appears to have on
the one sever I use it on.
and here are what I found in the savegrp.log the datasize is not the same,
but the # of files are larger,
Oct 12 03:15:21 ldap1.dk: tsproj: d:\Toldprojektet level=full,
638 MB 00:27:46 203393 files
Oct 18 01:42:57 ldap1.dk: tsproj: d:\Toldprojektet level=incr,
23 MB 00:00:57 400 files
I know it not possible to use my data as evidence in your enviorment.
I don't know if u have tried to enable change journal and compared results
but I would say it's worth a try.
**************************************************
regards
Kenneth Larsen
Steria
Tonsbakken 16-18
2740 Skovlunde
kel AT steria DOT dk - 44506261 - 26306261
**************************************************
|---------+--------------------------------->
| | Calvin Thomas |
| | <calvin.thomas@NACALOG|
| | ISTICS.COM> |
| | Sent by: Legato |
| | NetWorker discussion |
| | <NETWORKER AT LISTMAIL DOT TE|
| | MPLE.EDU> |
| | |
| | |
| | 17-10-2002 22:32 |
| | Please respond to |
| | Calvin Thomas |
| | |
|---------+--------------------------------->
>--------------------------------------------------------------------------------------------------------------|
|
|
| To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
|
| cc:
|
| Subject: [Networker] Resolution: Re: [Networker] What is the best
way to backup 100,000 files |
| in asingle subdirectory on a W2K Client
|
>--------------------------------------------------------------------------------------------------------------|
I have a solution that works for me.
Tar.exe (for dos) will create a 4.5GB tar file in another directory in
2
hours. Oddly enough, it was written to run under DOS3.0, but it still
works. It only uses 100KB of memory, and 3% of the processor time to read,
and merge all 100,000 files into a single file.
Kind of makes you wonder what the Networker client is
doing..............
To be fair, the tar command can't update the existing archive because of an
"out of memory" error, but I can delete and create a new tar archive every
day in two hours, so I don't mind very much. I have created a batch file
to
delete the existing archive, and create the new one and scheduled it to run
before the nightly backup.
Thanks for all the suggestions, and help on this issue. If anyone has any
further questions or thoughts feel free to contact me.
Thanks
Calvin E. Thomas
UNIX System Administrator
NACA Logistics.
----- Original Message -----
From: "Shelley L. Shostak" <sls AT qstech DOT com>
To: "Calvin Thomas" <calvin.thomas AT nacalogistics DOT com>
Sent: Wednesday, October 16, 2002 9:38 AM
Subject: Re: [Networker] What is the best way to backup 100,000 files in
asingle subdirectory on a W2K Client
> On Tue, 15 Oct 2002, Calvin Thomas wrote:
>
> >
> > Hi Shelley.
> > I also work in Unix, but haven't spent the time to become aquainted
with
> > perl yet.
> > I am not crazy about winzip, because if the number of files increase
> > past
> > 65K, it will fail again. I would rather work on a long term solution.
> > I am currently trying a save command from a dos window on the client.
I
> > am
> > disgusted to see that the save.exe command is using 50% processor power
> > for
> > 1+ hour (one of our two P3-600 processors) , has made 17,118 File
reads,
> > and
> > has not yet started to send any data to the networker server. To save
> > 89,000 files (we have paired them down some), this means 1 file read
for
> > every 6 files, and still working. I suspect it will time out before it
> > is
> > ready to send any data....
> > Maybe a script, (or perl) is the way to go, but I still have to reduce
> > the
> > number of files I backup to satisfy Networker requirements. I was
> > thinking
> > to try PKZip or some other third party command line driven program to
do
> > this.
> >
> > Have you tried anything like this?
> >
>
> Calvin,
>
> PKZip should work OK for this. I have not used it but I know people who
used
> it to package up software. Processing large directories on any platform
is
> difficult! That is a lot of files in one directory.
>
> Shelley
>
--
Note: To sign off this list, send a "signoff" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
--
Note: To sign off this list, send a "signoff" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
|
