[Veritas-bu] Millions of files in one directory

This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

------_=_NextPart_001_01C46510.D331F92A
Content-Type: text/plain
Content-Transfer-Encoding: 7bit

We've encountered similar problems here but not to that degree.  1-2 million
files is nuts.

The best solution I've seen is splitting up the files into manageable chunks
and making each one a separate stream, but that can have it's own issues if
it's a dynamic file list.  Sounds to me like the application design team for
this server didn't consider a few things.  Windows is notorious for its
dismal performance with a large number of small files, 3-4 times worse than
Unix.

 

 

-----Original Message-----
From: veritas-bu-admin AT mailman.eng.auburn DOT edu
[mailto:veritas-bu-admin AT mailman.eng.auburn DOT edu] On Behalf Of WILLIAMS,
KRISTOPHER L
Sent: Thursday, July 08, 2004 9:56
To: Netbackup User_Group
Subject: [Veritas-bu] Millions of files in one directory

 

Hello,

 

We have a client that is failing because there are too many small files in
one directory. There are apprx. 1 - 2 million files in two different
directories that are causing the failures. The system is a Win2k server
using NetBackup 5.0, master and media servers are also Win2K with NBU 5.0.
Has anyone had this problem and found a good resolution? 

 

 

Thanks,

 

Kris


------_=_NextPart_001_01C46510.D331F92A
Content-Type: text/html
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<html>

<head>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Dus-ascii">


<meta name=3DGenerator content=3D"Microsoft Word 10 (filtered)">
<title>RE: [Veritas-bu] Cleaining Tapes LTO's-2</title>

<style>
<!--
 /* Font Definitions */
 @font-face
        {font-family:Tahoma;
        panose-1:2 11 6 4 3 5 4 4 2 4;}
 /* Style Definitions */
 p.MsoNormal, li.MsoNormal, div.MsoNormal
        {margin:0in;
        margin-bottom:.0001pt;
        font-size:12.0pt;
        font-family:"Times New Roman";}
a:link, span.MsoHyperlink
        {color:blue;
        text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
        {color:blue;
        text-decoration:underline;}
p.MsoAutoSig, li.MsoAutoSig, div.MsoAutoSig
        {margin:0in;
        margin-bottom:.0001pt;
        font-size:12.0pt;
        font-family:"Times New Roman";}
p
        {margin-right:0in;
        margin-left:0in;
        font-size:12.0pt;
        font-family:"Times New Roman";}
span.emailstyle18
        {font-family:Arial;
        color:navy;}
span.EmailStyle19
        {font-family:Arial;
        color:navy;}
@page Section1
        {size:8.5in 11.0in;
        margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
        {page:Section1;}
-->
</style>

</head>

<body lang=3DEN-US link=3Dblue vlink=3Dblue>

<div class=3DSection1>

<p class=3DMsoNormal><font size=3D3 color=3Dnavy face=3D"Times New =
Roman"><span
style=3D'font-size:12.0pt;color:navy'>We've encountered similar =
problems
here but not to that degree.&nbsp; 1-2 million files is =
nuts.</span></font></p>

<p class=3DMsoNormal><font size=3D3 color=3Dnavy face=3D"Times New =
Roman"><span
style=3D'font-size:12.0pt;color:navy'>The best solution I've seen is
splitting up the files into manageable chunks and making each one a =
separate stream,
but that can have it's own issues if it's a dynamic file list.&nbsp;
Sounds to me like the application design team for this server didn't
consider a few things.&nbsp; Windows is notorious for its dismal =
performance with
a large number of small files, 3-4 times worse than =
Unix.</span></font></p>

<p class=3DMsoNormal><font size=3D3 color=3Dnavy face=3D"Times New =
Roman"><span
style=3D'font-size:12.0pt;color:navy'>&nbsp;</span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>&nbsp;</span></font></p>

<p class=3DMsoNormal style=3D'margin-left:.5in'><font size=3D2 =
face=3DTahoma><span
style=3D'font-size:10.0pt;font-family:Tahoma'>-----Original =
Message-----<br>
<b><span style=3D'font-weight:bold'>From:</span></b>
veritas-bu-admin AT mailman.eng.auburn DOT edu
[mailto:veritas-bu-admin AT mailman.eng.auburn DOT edu] <b><span =
style=3D'font-weight:
bold'>On Behalf Of </span></b>WILLIAMS, KRISTOPHER L<br>
<b><span style=3D'font-weight:bold'>Sent:</span></b> =
</span></font><font size=3D2 face=3DTahoma><span =
style=3D'font-size:10.0pt;font-family:Tahoma'>Thursday, July
 08, 2004</span></font><font size=3D2 face=3DTahoma><span =
style=3D'font-size:10.0pt;
font-family:Tahoma'> </span></font><font size=3D2 face=3DTahoma><span
 style=3D'font-size:10.0pt;font-family:Tahoma'>9:56</span></font><font =
size=3D2
face=3DTahoma><span style=3D'font-size:10.0pt;font-family:Tahoma'><br>
<b><span style=3D'font-weight:bold'>To:</span></b> Netbackup =
User_Group<br>
<b><span style=3D'font-weight:bold'>Subject:</span></b> [Veritas-bu] =
Millions of
files in one directory</span></font></p>

<p class=3DMsoNormal style=3D'margin-left:.5in'><font size=3D3 =
face=3D"Times New Roman"><span
style=3D'font-size:12.0pt'>&nbsp;</span></font></p>

<p style=3D'margin-left:1.0in'><font size=3D3 color=3Dnavy =
face=3D"Times New Roman"><span
style=3D'font-size:12.0pt;color:navy'>Hello,</span></font></p>

<p style=3D'margin-left:1.0in'><font size=3D3 color=3Dnavy =
face=3D"Times New Roman"><span
style=3D'font-size:12.0pt;color:navy'>&nbsp;</span></font></p>

<p style=3D'margin-left:1.0in'><font size=3D3 color=3Dnavy =
face=3D"Times New Roman"><span
style=3D'font-size:12.0pt;color:navy'>We have a client that is failing =
because
there are too many small files in one directory. There are apprx. 1 - 2
million files in two different directories that are causing the =
failures. The
system is a Win2k server using NetBackup 5.0, master and media servers =
are also
Win2K with NBU 5.0. Has anyone had this problem and found a good =
resolution?</span></font>
</p>

<p style=3D'margin-left:.5in'><font size=3D2 color=3Dnavy =
face=3DArial><span
style=3D'font-size:10.0pt;font-family:Arial;color:navy'>&nbsp;</span></f=
ont></p>

<p style=3D'margin-left:.5in'><font size=3D2 color=3Dnavy =
face=3DArial><span
style=3D'font-size:10.0pt;font-family:Arial;color:navy'>&nbsp;</span></f=
ont></p>

<p style=3D'margin-left:.5in'><font size=3D2 color=3Dnavy =
face=3DArial><span
style=3D'font-size:10.0pt;font-family:Arial;color:navy'>Thanks,</span></=
font></p>

<p style=3D'margin-left:.5in'><font size=3D2 color=3Dnavy =
face=3DArial><span
style=3D'font-size:10.0pt;font-family:Arial;color:navy'>&nbsp;</span></f=
ont></p>

<p style=3D'margin-left:.5in'><font size=3D2 color=3Dnavy =
face=3DArial><span
style=3D'font-size:10.0pt;font-family:Arial;color:navy'>Kris</span></fon=
t></p>

</div>

</body>

</html>

------_=_NextPart_001_01C46510.D331F92A--