|
[Veritas-bu] Backing up a large amount of small files
2002-11-02 12:22:06
This is a multipart message in MIME format.
--=_alternative 005F9D0B86256C65_=
Content-Type: text/plain; charset="us-ascii"
you mean raid 5 is great for reads but not for writes... :-)
writing has the extra overhead of calculating parity and then writing the
parity...
- Scott
"David A. Chapa" <david AT datastaff DOT com>
Sent by: veritas-bu-admin AT mailman.eng.auburn DOT edu
10/30/2002 11:03 AM
To: Bruno.Bossier AT comparex DOT be
cc: veritas-bu AT mailman.eng.auburn DOT edu
Subject: Re: [Veritas-bu] Backing up a large amount of small
files
Well, I did see something like this at a client and I always found the
best
solution to something like this is to just format the dang thing :-)
But Seriously now...
What RAID level, 5?
RAID5 is great for writes but not for reads, and with as many reads as you
need
done for this backup you would definitely hit timeouts. This is exactly
the
problem one of my current clients is facing.
What about the last time SCANDISK was run? We found a heavily fragmented
disk
caused these types of timeouts as well.
Some of the other things we tried was to break up the server backup at the
file
list. This was a bit more labor intensive, but it did work to some
degree.
HTH
David
Quoting Bruno.Bossier AT comparex DOT be:
> We have a server with a directory which contains over 3 million files
for a
> total amount of nearly 150 Gb. We are trying to back this up in a
> reasonable amount of time (12 to 14 hours maximum). OTM is enabled. We
have
> not succeeded so far. A lot of time is lost at the beginning of the
backup
> when the backup is going through the directory structure to check all
> files. This takes several hours. The first we already did until now is
to
> set the maximum cache size to 0 (unlimited). This stopped the errors we
saw
> in the Windows eventlog when OTM is started and then aborted after 40
> minutes, then restarted again ....
>
> Can someone give some suggestions to speed up this backup ?
>
> Thanks,
> Bruno
>
>
> _______________________________________________
> Veritas-bu maillist - Veritas-bu AT mailman.eng.auburn DOT edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
>
_______________________________________________
Veritas-bu maillist - Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
--=_alternative 005F9D0B86256C65_=
Content-Type: text/html; charset="us-ascii"
<br><font size=2 face="Arial">you mean raid 5 is great for reads but not for
writes... :-)</font>
<br>
<br><font size=2 face="Arial">writing has the extra overhead of calculating
parity and then writing the parity...</font>
<br>
<br>
<br><font size=2 face="Arial">- Scott</font>
<br>
<br>
<br>
<br>
<table width=100%>
<tr valign=top>
<td>
<td><font size=1 face="sans-serif"><b>"David A. Chapa" <david AT
datastaff DOT com></b></font>
<br><font size=1 face="sans-serif">Sent by: veritas-bu-admin AT
mailman.eng.auburn DOT edu</font>
<p><font size=1 face="sans-serif">10/30/2002 11:03 AM</font>
<br>
<td><font size=1 face="Arial"> </font>
<br><font size=1 face="sans-serif"> To:
Bruno.Bossier AT comparex DOT be</font>
<br><font size=1 face="sans-serif"> cc:
veritas-bu AT mailman.eng.auburn DOT edu</font>
<br><font size=1 face="sans-serif"> Subject:
Re: [Veritas-bu] Backing up a large amount of small
files</font></table>
<br>
<br>
<br><font size=2 face="Courier New">Well, I did see something like this at a
client and I always found the best <br>
solution to something like this is to just format the dang thing :-)<br>
<br>
But Seriously now...<br>
<br>
What RAID level, 5?<br>
<br>
RAID5 is great for writes but not for reads, and with as many reads as you need
<br>
done for this backup you would definitely hit timeouts. This is exactly
the <br>
problem one of my current clients is facing.<br>
<br>
What about the last time SCANDISK was run? We found a heavily fragmented
disk <br>
caused these types of timeouts as well.<br>
<br>
Some of the other things we tried was to break up the server backup at the file
<br>
list. This was a bit more labor intensive, but it did work to some
degree.<br>
<br>
HTH<br>
<br>
David<br>
<br>
Quoting Bruno.Bossier AT comparex DOT be:<br>
<br>
> We have a server with a directory which contains over 3 million files for
a<br>
> total amount of nearly 150 Gb. We are trying to back this up in a<br>
> reasonable amount of time (12 to 14 hours maximum). OTM is enabled. We
have<br>
> not succeeded so far. A lot of time is lost at the beginning of the
backup<br>
> when the backup is going through the directory structure to check all<br>
> files. This takes several hours. The first we already did until now is
to<br>
> set the maximum cache size to 0 (unlimited). This stopped the errors we
saw<br>
> in the Windows eventlog when OTM is started and then aborted after 40<br>
> minutes, then restarted again ....<br>
> <br>
> Can someone give some suggestions to speed up this backup ?<br>
> <br>
> Thanks,<br>
> Bruno<br>
> <br>
> <br>
> _______________________________________________<br>
> Veritas-bu maillist - Veritas-bu AT mailman.eng.auburn DOT
edu<br>
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu<br>
> <br>
<br>
<br>
<br>
_______________________________________________<br>
Veritas-bu maillist - Veritas-bu AT mailman.eng.auburn DOT edu<br>
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu<br>
</font>
<br>
<br>
--=_alternative 005F9D0B86256C65_=--
|
|
|
