Veritas-bu

[Veritas-bu] Backing up a large amount of small files

2002-11-02 14:33:36
Subject: [Veritas-bu] Backing up a large amount of small files
From: scott.kendall AT abbott DOT com (scott.kendall AT abbott DOT com)
Date: Sat, 2 Nov 2002 13:33:36 -0600
This is a multipart message in MIME format.
--=_alternative 006BA6CF86256C65_=
Content-Type: text/plain; charset="us-ascii"

yes, this will also work on windows, sort of... it works a little 
different though.  search the list archives (i think it's came up before) 
or look at technet.  it's a registry setting for the file system.

on unix, you're telling netbackup to not look at the last access or care 
about setting it back when it's done.  this reduces the amount of 
instructions performed by netbackup and the number of writes to the file 
system.

on windows, you're telling the os not to adjust the last access time on 
the filesystem.  i think netbackup still looks at the last access time, 
but it doesn't need to set it back after it's backed up the file because 
it sees it's still the same.  this reduces the instructions performed by 
the os for each file read as well as the write to the file system to set 
the time back to the original.


- Scott





"Marx, Keath" <kmarx AT trigon DOT com>
Sent by: veritas-bu-admin AT mailman.eng.auburn DOT edu
10/30/2002 03:24 PM

 
        To:     "'veritas-bu AT mailman.eng.auburn DOT edu'" <veritas-bu AT 
mailman.eng.auburn DOT edu>
        cc: 
        Subject:        RE: [Veritas-bu] Backing up a large amount of small 
files


I have been searching but can't seem to find any indication one way or the
other.  Does this issue impact win2k servers as well?

-----Original Message-----
From: Nash, Ebon [mailto:Ebon_Nash AT compuware DOT com] 
Sent: Wednesday, October 30, 2002 1:47 PM
To: 'paulg AT CDCNA DOT COM'; Bruno.Bossier AT comparex DOT be;
veritas-bu AT mailman.eng.auburn DOT edu
Subject: RE: [Veritas-bu] Backing up a large amount of small files

There is a Veritas Technote that discusses poor backup throughput
performance on RAID 5 disks.  It shows how to disable the resetting of
access times which can seriously degrade performance on machines with a 
lot
of smaller files.  See technote 240607 on Veritas' support site for 
further
information.


Ebon

-----Original Message-----
From: paulg AT CDCNA DOT COM [mailto:paulg AT CDCNA DOT COM]
Sent: Wednesday, October 30, 2002 1:39 PM
To: Bruno.Bossier AT comparex DOT be; veritas-bu AT mailman.eng.auburn DOT edu
Subject: Re: [Veritas-bu] Backing up a large amount of small files


You might want to consider OFM from StBernard, it definitely has an edge
over
OTM.  Should run you about $600/license.  While not without issues, file
checkpointing capabilities of OFM are definitely better than that of OTM.

paulg

On Oct 30,  5:36pm, <Bruno.Bossier AT comparex DOT be> wrote:
> Subject: [Veritas-bu] Backing up a large amount of small files
> We have a server with a directory which contains over 3 million files 
for
a
> total amount of nearly 150 Gb. We are trying to back this up in a
> reasonable amount of time (12 to 14 hours maximum). OTM is enabled. We
have
> not succeeded so far. A lot of time is lost at the beginning of the 
backup
> when the backup is going through the directory structure to check all
> files. This takes several hours. The first we already did until now is 
to
> set the maximum cache size to 0 (unlimited). This stopped the errors we
saw
> in the Windows eventlog when OTM is started and then aborted after 40
> minutes, then restarted again ....
>
> Can someone give some suggestions to speed up this backup ?
>
> Thanks,
> Bruno
>
>
> _______________________________________________
> Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
>-- End of excerpt from <Bruno.Bossier AT comparex DOT be>


_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu



The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or 
disclose
it to anyone else. If you received it in error please notify us 
immediately
and then destroy it. 

_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu



--=_alternative 006BA6CF86256C65_=
Content-Type: text/html; charset="us-ascii"


<br><font size=2 face="Arial">yes, this will also work on windows, sort of... 
it works a little different though. &nbsp;search the list archives (i think 
it's came up before) or look at technet. &nbsp;it's a registry setting for the 
file system.</font>
<br>
<br><font size=2 face="Arial">on unix, you're telling netbackup to not look at 
the last access or care about setting it back when it's done. &nbsp;this 
reduces the amount of instructions performed by netbackup and the number of 
writes to the file system.</font>
<br>
<br><font size=2 face="Arial">on windows, you're telling the os not to adjust 
the last access time on the filesystem. &nbsp;i think netbackup still looks at 
the last access time, but it doesn't need to set it back after it's backed up 
the file because it sees it's still the same. &nbsp;this reduces the 
instructions performed by the os for each file read as well as the write to the 
file system to set the time back to the original.</font>
<br>
<br>
<br><font size=2 face="Arial">- Scott</font>
<br>
<br>
<br>
<br>
<table width=100%>
<tr valign=top>
<td>
<td><font size=1 face="sans-serif"><b>&quot;Marx, Keath&quot; &lt;kmarx AT 
trigon DOT com&gt;</b></font>
<br><font size=1 face="sans-serif">Sent by: veritas-bu-admin AT 
mailman.eng.auburn DOT edu</font>
<p><font size=1 face="sans-serif">10/30/2002 03:24 PM</font>
<br>
<td><font size=1 face="Arial">&nbsp; &nbsp; &nbsp; &nbsp; </font>
<br><font size=1 face="sans-serif">&nbsp; &nbsp; &nbsp; &nbsp; To: &nbsp; 
&nbsp; &nbsp; &nbsp;&quot;'veritas-bu AT mailman.eng.auburn DOT edu'&quot; 
&lt;veritas-bu AT mailman.eng.auburn DOT edu&gt;</font>
<br><font size=1 face="sans-serif">&nbsp; &nbsp; &nbsp; &nbsp; cc: &nbsp; 
&nbsp; &nbsp; &nbsp;</font>
<br><font size=1 face="sans-serif">&nbsp; &nbsp; &nbsp; &nbsp; Subject: &nbsp; 
&nbsp; &nbsp; &nbsp;RE: [Veritas-bu] Backing up a large amount of small 
files</font></table>
<br>
<br>
<br><font size=2 face="Courier New">I have been searching but can't seem to 
find any indication one way or the<br>
other. &nbsp;Does this issue impact win2k servers as well?<br>
<br>
-----Original Message-----<br>
From: Nash, Ebon [mailto:Ebon_Nash AT compuware DOT com] <br>
Sent: Wednesday, October 30, 2002 1:47 PM<br>
To: 'paulg AT CDCNA DOT COM'; Bruno.Bossier AT comparex DOT be;<br>
veritas-bu AT mailman.eng.auburn DOT edu<br>
Subject: RE: [Veritas-bu] Backing up a large amount of small files<br>
<br>
There is a Veritas Technote that discusses poor backup throughput<br>
performance on RAID 5 disks. &nbsp;It shows how to disable the resetting of<br>
access times which can seriously degrade performance on machines with a lot<br>
of smaller files. &nbsp;See technote 240607 on Veritas' support site for 
further<br>
information.<br>
<br>
<br>
Ebon<br>
<br>
-----Original Message-----<br>
From: paulg AT CDCNA DOT COM [mailto:paulg AT CDCNA DOT COM]<br>
Sent: Wednesday, October 30, 2002 1:39 PM<br>
To: Bruno.Bossier AT comparex DOT be; veritas-bu AT mailman.eng.auburn DOT 
edu<br>
Subject: Re: [Veritas-bu] Backing up a large amount of small files<br>
<br>
<br>
You might want to consider OFM from StBernard, it definitely has an edge<br>
over<br>
OTM. &nbsp;Should run you about $600/license. &nbsp;While not without issues, 
file<br>
checkpointing capabilities of OFM are definitely better than that of OTM.<br>
<br>
paulg<br>
<br>
On Oct 30, &nbsp;5:36pm, &lt;Bruno.Bossier AT comparex DOT be&gt; wrote:<br>
&gt; Subject: [Veritas-bu] Backing up a large amount of small files<br>
&gt; We have a server with a directory which contains over 3 million files 
for<br>
a<br>
&gt; total amount of nearly 150 Gb. We are trying to back this up in a<br>
&gt; reasonable amount of time (12 to 14 hours maximum). OTM is enabled. We<br>
have<br>
&gt; not succeeded so far. A lot of time is lost at the beginning of the 
backup<br>
&gt; when the backup is going through the directory structure to check all<br>
&gt; files. This takes several hours. The first we already did until now is 
to<br>
&gt; set the maximum cache size to 0 (unlimited). This stopped the errors we<br>
saw<br>
&gt; in the Windows eventlog when OTM is started and then aborted after 40<br>
&gt; minutes, then restarted again ....<br>
&gt;<br>
&gt; Can someone give some suggestions to speed up this backup ?<br>
&gt;<br>
&gt; Thanks,<br>
&gt; Bruno<br>
&gt;<br>
&gt;<br>
&gt; _______________________________________________<br>
&gt; Veritas-bu maillist &nbsp;- &nbsp;Veritas-bu AT mailman.eng.auburn DOT 
edu<br>
&gt; http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu<br>
&gt;-- End of excerpt from &lt;Bruno.Bossier AT comparex DOT be&gt;<br>
<br>
<br>
_______________________________________________<br>
Veritas-bu maillist &nbsp;- &nbsp;Veritas-bu AT mailman.eng.auburn DOT edu<br>
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu<br>
<br>
<br>
<br>
The contents of this e-mail are intended for the named addressee only. It<br>
contains information that may be confidential. Unless you are the named<br>
addressee or an authorized designee, you may not copy or use it, or disclose<br>
it to anyone else. If you received it in error please notify us immediately<br>
and then destroy it. <br>
<br>
_______________________________________________<br>
Veritas-bu maillist &nbsp;- &nbsp;Veritas-bu AT mailman.eng.auburn DOT edu<br>
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu<br>
_______________________________________________<br>
Veritas-bu maillist &nbsp;- &nbsp;Veritas-bu AT mailman.eng.auburn DOT 
edu</font>
<br><font size=2 face="Courier 
New">http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu<br>
</font>
<br>
<br>
--=_alternative 006BA6CF86256C65_=--