Veritas-bu

[Veritas-bu] Backing up a large amount of small files

2002-11-02 05:43:52
Subject: [Veritas-bu] Backing up a large amount of small files
From: joe AT joe DOT net (Johnny Oestergaard)
Date: Sat, 02 Nov 2002 11:43:52 +0100
We have the same type of problem and except that we have turned OTM off and 
have less files the only way I see that we can solve the problem is by 
dividing the directories into different streems and streem the backup to 
disk (so that it doesn't hold a tapedrive for days)
Later we will move the backup from disk to tape but that should happen at a 
speed that we can handle.

It's great to have so fast tapedrives as we have today (if NT/Win2K could 
pump the data at that speed)

Our files are on a StorageTek SVA9500 and it's still so slow that we cry. 
It's not the disks that are the problem but the time it takes for the 
operatingsystem to open and close the files.
It would also be a problem on any other operatingsystem (but could be it 
would be a little smaller)

I have been thinking if NBU could take a physical backup of the drive 
(track by track) This should be fast but would also backup empty tracks but 
at 30MB/s or more who cares.

/johnny

At 12:03 30-10-2002 -0500, David A. Chapa wrote:
>Well, I did see something like this at a client and I always found the best
>solution to something like this is to just format the dang thing :-)
>
>But Seriously now...
>
>What RAID level, 5?
>
>RAID5 is great for writes but not for reads, and with as many reads as you 
>need
>done for this backup you would definitely hit timeouts.  This is exactly the
>problem one of my current clients is facing.
>
>What about the last time SCANDISK was run?  We found a heavily fragmented 
>disk
>caused these types of timeouts as well.
>
>Some of the other things we tried was to break up the server backup at the 
>file
>list.  This was a bit more labor intensive, but it did work to some degree.
>
>HTH
>
>David
>
>Quoting Bruno.Bossier AT comparex DOT be:
>
> > We have a server with a directory which contains over 3 million files for a
> > total amount of nearly 150 Gb. We are trying to back this up in a
> > reasonable amount of time (12 to 14 hours maximum). OTM is enabled. We have
> > not succeeded so far. A lot of time is lost at the beginning of the backup
> > when the backup is going through the directory structure to check all
> > files. This takes several hours. The first we already did until now is to
> > set the maximum cache size to 0 (unlimited). This stopped the errors we saw
> > in the Windows eventlog when OTM is started and then aborted after 40
> > minutes, then restarted again ....
> >
> > Can someone give some suggestions to speed up this backup ?
> >
> > Thanks,
> > Bruno
> >
> >
> > _______________________________________________
> > Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
> > http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
> >
>
>
>
>_______________________________________________
>Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
>http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu


<Prev in Thread] Current Thread [Next in Thread>