I had a nt4 server with in excess of 400K files in a directory structure,
the only solution I found was to increase inactivity time out... BUT on W2K
you have the option of using change journal, which if memory serves me
right should hold the solution to your problem, since networker will talk
to change journal, who since the last backup have kept an eye out on files
changed..... I'm no wizard on change journal, but I think if u configure
and enable change journal, it should be installed default on w2k...
**************************************************
regards
Kenneth Larsen
Steria
Tonsbakken 16-18
2740 Skovlunde
kel AT steria DOT dk - 44506261 - 26306261
**************************************************
|---------+--------------------------------->
| | Calvin Thomas |
| | <calvin.thomas@NACALOG|
| | ISTICS.COM> |
| | Sent by: Legato |
| | NetWorker discussion |
| | <NETWORKER AT LISTMAIL DOT TE|
| | MPLE.EDU> |
| | |
| | |
| | 16-10-2002 09:28 |
| | Please respond to |
| | Calvin Thomas |
| | |
|---------+--------------------------------->
>--------------------------------------------------------------------------------------------------------------|
|
|
| To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
|
| cc:
|
| Subject: [Networker] UPDATE:Re: [Networker] What is the best way to
backup 100,000 files in a |
| single subdirectory on a W2K Client
|
>--------------------------------------------------------------------------------------------------------------|
My problem isn't with the save streams that start... They are running at
1/2 the speed of my DLT4000 anyway.
As far as defragmenting the file system, this is a fax server that replaces
these 100,000 file in 5 days. It is a good thought but I don't think it
would help much because the real problem is with the save streams that
don't
start.
I have an update on this part of the problem:
I started a save session of the subdirectory from the command line on the
client, and opened task manager to watch save.exe perform. It has been 10
hours since I started the save session, and it is still cranking away as I
write this. So far, in 10 hours it has accessed the disk drive (I/O Reads)
46,486 times, using 50% processor time, and shows no sign of slowing up. I
figure that the client is accessing the disk one time for each file it will
be backing up. The speed seems to be about 1 file access per second, so
that
means 100,000 files = 100,000 seconds, or 27 hours.
Kind of a long time to start a backup session........
I guess this means that save.exe can't do the job no matter how I configure
it. I will have to look for a third party solution like the zip or rar
utilities to zip all these files up to a single file that save.exe can
handle in a reasonable amount of time.
If anyone see anything I missed, please let me know.
Thanks
Calvin E. Thomas
----- Original Message -----
From: <lemons_terry AT emc DOT com>
To: <calvin.thomas AT nacalogistics DOT com>
Cc: <NETWORKER AT LISTMAIL.TEMPLE DOT EDU>
Sent: Tuesday, October 15, 2002 8:43 PM
Subject: RE: [Networker] What is the best way to backup 100,000 files in a
single subdirectory on a W2K Client
> Hi Calvin
>
> My gut feel is that 12 simultaneous streams might be too many. Unless
you
> have a very high speed file system and disk subsystem, I'd cut that
number
> at least in half.
>
> By the by, has the file system holding these 100,000 files been
defragmented
> recently? Fragmentation certainly won't help the backup performance.
>
> Thanks
> tl
>
> -----Original Message-----
> From: Calvin Thomas [mailto:calvin.thomas AT nacalogistics DOT com]
> Sent: Tuesday, October 15, 2002 2:44 PM
> To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
> Subject: Re: [Networker] What is the best way to backup 100,000 files in
> a single subdirectory on a W2K Client
>
>
> Most of the streams start (after a variable amount of time). The
*.302,
> *.303,*.304 Etc up to *.320 all start. I have seen 12 streams running
from
> this client at one time, but the *.301 doesn't start since it has 1/2 the
> files in it. It appears the client doesn't have enough time to select
all
> 60K files to backup before the 90 minute timeout. I can't raise the
timeout
> value because it extends out of our backup window.
>
> I know it isn't the tape drive, because I am backing up to a hard
drive.
> This eliminates such problems as shoe-shining, and contention between
> clients.
>
> What I seem to need is a way of archiveing the files on the client to
a
> single file, that I can then backup in place of this subdirectory. I
tried
> winzip to create a zip archive, but it is limited to 65K files.
> Any other ideas?
> Calvin Thomas
>
>
>
> ----- Original Message -----
> From: "Patrick Sibilio" <patrick.sibilio AT COMMERZBANK DOT COM>
> To: <NETWORKER AT LISTMAIL.TEMPLE DOT EDU>
> Sent: Tuesday, October 15, 2002 1:01 AM
> Subject: Re: [Networker] What is the best way to backup 100,000 files in
a
> single subdirectory on a W2K Client
>
>
> > Hi,
> >
> > how many streams are acitve during the backup? How many clients are in
the
> > group?
> >
> > Try backing up the client with a new group and a different start time.
> >
> > regards,
> > Patrick Sibilio
> >
> > On Mon, 14 Oct 2002 18:10:00 -0700, Calvin Thomas
> > <calvin.thomas AT NACALOGISTICS DOT COM> wrote:
> >
> > >I have tried seperate save streams by specifying
> > >
> > >C:\Directory\*.301
> > >C:\Directory\*.302
> > >C:\Directory\*.303
> > >Etc.
> > >The problem is the biggest save stream (c:\directory\*.301) never
starts
> > >even though I have set the time out to 90Minutes. I can't set the time
> out
> > >any longer due to other considerations.
> > >When I back up using ALL, the writes happen at 32-64KBPS, and the
backup
> > >takes 20 hours.
> > >I tried to create a zip file using WinZip, but WinZip is limited to
65K
> > >files.
> > >
> > >What have your experiences been in similar sittuations?
> > >
> > >Thanks
> > >Calvin Thomas
> > >UNIX system administrator
> > >NACA Logistics
> > >
> > >--
> > >Note: To sign off this list, send a "signoff" command via email
> > >to listserv AT listmail.temple DOT edu or visit the list's Web site at
> > >http://listmail.temple.edu/archives/networker.html where you can
> > >also view and post messages to the list.
> > >=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
> >
> > --
> > Note: To sign off this list, send a "signoff" command via email
> > to listserv AT listmail.temple DOT edu or visit the list's Web site at
> > http://listmail.temple.edu/archives/networker.html where you can
> > also view and post messages to the list.
> > =*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
> >
>
> --
> Note: To sign off this list, send a "signoff" command via email
> to listserv AT listmail.temple DOT edu or visit the list's Web site at
> http://listmail.temple.edu/archives/networker.html where you can
> also view and post messages to the list.
> =*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
>
--
Note: To sign off this list, send a "signoff" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
--
Note: To sign off this list, send a "signoff" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
|
