hi,

I have had similar problems with a UNIX client holding a couple of
hundred thousand files.
( Mentor/Synopsis CAD seems to create files at abandon. )

And as you say the problem is not the speed of backup, but rather
that the time it takes to determine which files to backup is too long.

My short term solution was to do full backups. Since every file needed
to be backed up, it didn't take long for the stream to start and pump data.

The long term solution was to go for a faster disk subsystem on the client.
( Mirrored stripe on external raid, with Veritas filesystem instead of
Solaris default 2.6 ufs. )
Still takes about an hour to do an incremental backup, but this is within
my backup window.

Hope it helps some.

Regards,
Tommy Lindqvist

At 11:43 2002-10-15 -0700, Calvin Thomas wrote:
>    Most of the streams start (after a variable amount of time).  The *.302,
>*.303,*.304 Etc up to *.320 all start.  I have seen 12 streams running from
>this client at one time, but the *.301 doesn't start since it has 1/2 the
>files in it.  It appears the client doesn't have enough time to select all
>60K files to backup before the 90 minute timeout.  I can't raise the timeout
>value because it extends out of our backup window.
>
>    I know it isn't the tape drive, because I am backing up to a hard drive.
>This eliminates such problems as shoe-shining, and contention between
>clients.
>
>    What I seem to need is a way of archiveing the files on the client to a
>single file, that I can then backup in place of this subdirectory.  I tried
>winzip to create a zip archive, but it is limited to 65K files.
>Any other ideas?
>Calvin Thomas
>
>
>
>----- Original Message -----
>From: "Patrick Sibilio" <patrick.sibilio AT COMMERZBANK DOT COM>
>To: <NETWORKER AT LISTMAIL.TEMPLE DOT EDU>
>Sent: Tuesday, October 15, 2002 1:01 AM
>Subject: Re: [Networker] What is the best way to backup 100,000 files in a
>single subdirectory on a W2K Client
>
>
>> Hi,
>>
>> how many streams are acitve during the backup? How many clients are in the
>> group?
>>
>> Try backing up the client with a new group and a different start time.
>>
>> regards,
>> Patrick Sibilio
>>
>> On Mon, 14 Oct 2002 18:10:00 -0700, Calvin Thomas
>> <calvin.thomas AT NACALOGISTICS DOT COM> wrote:
>>
>> >I have tried seperate save streams by specifying
>> >
>> >C:\Directory\*.301
>> >C:\Directory\*.302
>> >C:\Directory\*.303
>> >Etc.
>> >The problem is the biggest save stream (c:\directory\*.301) never starts
>> >even though I have set the time out to 90Minutes. I can't set the time
>out
>> >any longer due to other considerations.
>> >When I back up using ALL, the writes happen at 32-64KBPS, and the backup
>> >takes 20 hours.
>> >I tried to create a zip file using WinZip, but WinZip is limited to 65K
>> >files.
>> >
>> >What have your experiences been in similar sittuations?
>> >
>> >Thanks
>> >Calvin Thomas
>> >UNIX system administrator
>> >NACA Logistics
>> >
>> >--
>> >Note: To sign off this list, send a "signoff" command via email
>> >to listserv AT listmail.temple DOT edu or visit the list's Web site at
>> >http://listmail.temple.edu/archives/networker.html where you can
>> >also view and post messages to the list.
>> >=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
>>
>> --
>> Note: To sign off this list, send a "signoff" command via email
>> to listserv AT listmail.temple DOT edu or visit the list's Web site at
>> http://listmail.temple.edu/archives/networker.html where you can
>> also view and post messages to the list.
>> =*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
>>
>
>--
>Note: To sign off this list, send a "signoff" command via email
>to listserv AT listmail.temple DOT edu or visit the list's Web site at
>http://listmail.temple.edu/archives/networker.html where you can
>also view and post messages to the list.
>=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
>
--
Saab Ericsson Space AB                  /"\
Systems manager   +46 (0)31 735 4391    \ /    ASCII Ribbon Campaign
tommy.lindqvist AT space DOT se                 X   against HTML email & vCards
postmaster                              / \    Keep Email as ASCII

--
Note: To sign off this list, send a "signoff" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=