[Veritas-bu] Backing up a large amount of small files
2002-11-05 09:03:37
That is probably the best thing to do.
Thanks !
Bruno
Rockey Reed
<Rockey.Reed@ve To: "'Bruno.Bossier AT comparex
DOT be'" <Bruno.Bossier AT comparex DOT be>
ritas.com> cc:
Subject: RE: [Veritas-bu] Backing
up a large amount of small files
11/05/2002
05:15
OK Bruno, let's be honest here. NBU as with all backup software
applications have difficulty with large quantities of small files. This is
caused by the overhead required to catalog the files, retain metadata of
their locations on tape, etcetera. A recommendation would be to look at
breaking the 150Gb into something like three 50GB chunks. Do this in the
file selection using the * wildcard and streaming the data. The file list
would look something like this:
NEW_STREAM
/path/a*
/path/b*
|
/path/g*
NEW_STREAM
/path/h*
|
/path/m*
down to /path/z*
To write to one drive set multiplexing in both the STU and the schedule and
set it to the number of streams.
HTH.
Thanks,
Rockey Reed
-----Original Message-----
From: Bruno.Bossier AT comparex DOT be [mailto:Bruno.Bossier AT comparex DOT be]
Sent: Monday, November 04, 2002 9:34 AM
To: David A. Chapa
Cc: veritas-bu AT mailman.eng.auburn DOT edu
Subject: Re: [Veritas-bu] Backing up a large amount of small files
All the data is on EMC via a SAN, so RAID-1.
"David A.
Chapa" To: Bruno.Bossier AT comparex DOT
be
<david@datasta cc:
veritas-bu AT mailman.eng.auburn DOT edu
ff.com> Subject: Re: [Veritas-bu]
Backing up a large amount of small files
10/30/2002
18:03
Well, I did see something like this at a client and I always found the best
solution to something like this is to just format the dang thing :-)
But Seriously now...
What RAID level, 5?
RAID5 is great for writes but not for reads, and with as many reads as you
need
done for this backup you would definitely hit timeouts. This is exactly
the
problem one of my current clients is facing.
What about the last time SCANDISK was run? We found a heavily fragmented
disk
caused these types of timeouts as well.
Some of the other things we tried was to break up the server backup at the
file
list. This was a bit more labor intensive, but it did work to some degree.
HTH
David
Quoting Bruno.Bossier AT comparex DOT be:
> We have a server with a directory which contains over 3 million files for
a
> total amount of nearly 150 Gb. We are trying to back this up in a
> reasonable amount of time (12 to 14 hours maximum). OTM is enabled. We
have
> not succeeded so far. A lot of time is lost at the beginning of the
backup
> when the backup is going through the directory structure to check all
> files. This takes several hours. The first we already did until now is to
> set the maximum cache size to 0 (unlimited). This stopped the errors we
saw
> in the Windows eventlog when OTM is started and then aborted after 40
> minutes, then restarted again ....
>
> Can someone give some suggestions to speed up this backup ?
>
> Thanks,
> Bruno
>
>
> _______________________________________________
> Veritas-bu maillist - Veritas-bu AT mailman.eng.auburn DOT edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
>
_______________________________________________
Veritas-bu maillist - Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
|
<Prev in Thread] |
Current Thread |
[Next in Thread>
|
- [Veritas-bu] Backing up a large amount of small files, Johnny Oestergaard
- [Veritas-bu] Backing up a large amount of small files, scott.kendall AT abbott DOT com
- [Veritas-bu] Backing up a large amount of small files, scott.kendall AT abbott DOT com
- [Veritas-bu] Backing up a large amount of small files, scott.kendall AT abbott DOT com
- [Veritas-bu] Backing up a large amount of small files, scott.kendall AT abbott DOT com
- [Veritas-bu] Backing up a large amount of small files, Johnny Oestergaard
- [Veritas-bu] Backing up a large amount of small files, scott.kendall AT abbott DOT com
- [Veritas-bu] Backing up a large amount of small files, Bruno.Bossier AT comparex DOT be
- [Veritas-bu] Backing up a large amount of small files,
Bruno.Bossier AT comparex DOT be <=
- [Veritas-bu] Backing up a large amount of small files, Vapiwala, Kevin V [IT]
- [Veritas-bu] Backing up a large amount of small files, Quarantine
- [Veritas-bu] Backing up a large amount of small files, Bruno.Bossier AT comparex DOT be
- [Veritas-bu] Backing up a large amount of small files, Bruno.Bossier AT comparex DOT be
|
|
|