Veritas-bu

[Veritas-bu] Netbackup Database size and speed

2000-10-18 04:33:39
Subject: [Veritas-bu] Netbackup Database size and speed
From: Mark Smiles msmiles AT lucent DOT com
Date: Wed, 18 Oct 2000 09:33:39 +0100
--------------080005010405040809080709
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit

I think you will find this is limited (for 3.2) to a maximum of 60 GB , 
this is within the capacity of Quantum  DLT 7000 (IV) @ 70 GB with 
compression turned on.
Regrds,
Mark Smiles
McMurphy, Tim wrote:
> 
> If anyone has any insight please post to the group! I am soon to re-design
> and implement our netbackup structure and one of the problems is, you
> guessed it. Our db is only 22 gig (small compared to some) but big enough to
> cause me concern for the future.
> 
> I was thinking along the lines of having one database per year (1999, 2000,
> 2001 etc) and trying to keep the size in line that way. Alternatively
> separate a separate db for different platforms (not my fav by a long
> stretch).
> 
> A question for you Jonathan is what do you backup your db to? I was under
> the impression that the netbackup db couldn't span multiple tapes. If I am
> wrong on that then good but if it is true then you must be near the limit of
> any tape technology that I am aware of.
> 
> -----Original Message-----
> From: Jonathan Geibel [mailto:Jonathan.Geibel AT disney DOT com]
> Sent: Tuesday, October 17, 2000 12:41 PM
> To: veritas-bu AT mailman.eng.auburn DOT edu
> Subject: [Veritas-bu] Netbackup Database size and speed
> 
> 
> Hello,
> 
> Question for you all:
> 
> Has anyone had any troubles with the netbackup database size getting out
> of hand and creating excessively bad performance issues?  Has anyone come
> up with any intuitive solutions for how to solve this problem?
> 
> With 13TBs of data being backed up every week, our netbackup db/images
> directory has grown to obscene sizes (60GB of flat-file data compressed).
> 
> Because netbackup doesn't use a real database to hold this information
> (simple flatfiles..) the scalability of this design is rather weak..
> 
> these huge flatfile databases are causing us major performance problems
> when trying to do simple restores..  the simple act of looking for a
> single file can take several hours to complete as it chugs through all of
> it's backups files.
> 
> we've indexed everything, but that doesn't seem to help..
> 
> I've heard there are tools to dump this information into an actual SQL
> database which could be an extremely useful tool for hunting down the
> backups for specific files..
> 
> has anyone done this?  we were thinking of possible dumping the data once
> a day into a database for doing quick searches on..
> 
> I'd be curious to hear if anyone else has run into these performance
> problems..
> 
> Jon
> 
> 
> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
> Walt Disney Feature Animation      The Secret Lab
> Senior Systems Engineer              818.526.3051
> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
> 
> 
> 
> 
> 
> _______________________________________________
> Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
> _______________________________________________
> Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
> 

-- 
------------------------------------------------------------------
Mark Smiles, Unix Systems Manager ME CIO EMEA
Lucent Technologies MicroElectronics Group      Kingswood,
Work Phone : +44 (0)1344 29 6042 + voicemail    Kings Ride,
Work Fax   : +44 (0)1344 29 6002                Ascot,
Email : msmiles AT lucent DOT com                       Berkshire, SL5 8AD

OK, who stopped payment on my reality check?
------------------------------------------------------------------
 


--------------080005010405040809080709
Content-Type: text/html; charset=us-ascii
Content-Transfer-Encoding: 7bit

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<html><head></head><body>I
 think you will find this is limited (for 3.2) to a maximum of 60 GB , this
 is within the capacity of Quantum&nbsp; DLT 7000 (IV) @ 70 GB with compression
 turned on.<br>
Regrds,<br>
Mark Smiles<br>
McMurphy, Tim wrote:<br>
<blockquote type="cite" cite="mid:E117645F1A1CD2118ACC00805FBBAA7C056046C3 AT 
mail.csd.canada.cdev DOT com">
<div class="text-plain"><pre wrap>If anyone has any insight please post to the 
group! I am soon to re-design
and implement our netbackup structure and one of the problems is, you
guessed it. Our db is only 22 gig (small compared to some) but big enough to
cause me concern for the future.

I was thinking along the lines of having one database per year (1999, 2000,
2001 etc) and trying to keep the size in line that way. Alternatively
separate a separate db for different platforms (not my fav by a long
stretch).

A question for you Jonathan is what do you backup your db to? I was under
the impression that the netbackup db couldn't span multiple tapes. If I am
wrong on that then good but if it is true then you must be near the limit of
any tape technology that I am aware of.

-----Original Message-----
From: Jonathan Geibel [<a class="txt-link txt-link-freetext" 
href="mailto:Jonathan.Geibel AT disney DOT com">mailto:Jonathan.Geibel AT 
disney DOT com</a>]
Sent: Tuesday, October 17, 2000 12:41 PM
To: <a class="txt-link txt-link-abbreviated" href="mailto:veritas-bu AT 
mailman.eng.auburn DOT edu">veritas-bu AT mailman.eng.auburn DOT edu</a>
Subject: [Veritas-bu] Netbackup Database size and speed


Hello,

Question for you all:

Has anyone had any troubles with the netbackup database size getting out
of hand and creating excessively bad performance issues?  Has anyone come
up with any intuitive solutions for how to solve this problem?

With 13TBs of data being backed up every week, our netbackup db/images
directory has grown to obscene sizes (60GB of flat-file data compressed).

Because netbackup doesn't use a real database to hold this information
(simple flatfiles..) the scalability of this design is rather weak..

these huge flatfile databases are causing us major performance problems
when trying to do simple restores..  the simple act of looking for a
single file can take several hours to complete as it chugs through all of
it's backups files.

we've indexed everything, but that doesn't seem to help..

I've heard there are tools to dump this information into an actual SQL
database which could be an extremely useful tool for hunting down the
backups for specific files..

has anyone done this?  we were thinking of possible dumping the data once
a day into a database for doing quick searches on..

I'd be curious to hear if anyone else has run into these performance
problems..

Jon


=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Walt Disney Feature Animation      The Secret Lab
Senior Systems Engineer              818.526.3051
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=





_______________________________________________
Veritas-bu maillist  -  <a class="txt-link txt-link-abbreviated" 
href="mailto:Veritas-bu AT mailman.eng.auburn DOT edu">Veritas-bu AT 
mailman.eng.auburn DOT edu</a>
<a class="txt-link txt-link-freetext" 
href="http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu";>http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu</a>
_______________________________________________
Veritas-bu maillist  -  <a class="txt-link txt-link-abbreviated" 
href="mailto:Veritas-bu AT mailman.eng.auburn DOT edu">Veritas-bu AT 
mailman.eng.auburn DOT edu</a>
<a class="txt-link txt-link-freetext" 
href="http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu";>http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu</a>
</pre></div>
</blockquote>
<br>
<div class="signature"><pre>-- 
------------------------------------------------------------------
Mark Smiles, Unix Systems Manager ME CIO EMEA
Lucent Technologies MicroElectronics Group      Kingswood,
Work Phone : +44 (0)1344 29 6042 + voicemail    Kings Ride,
Work Fax   : +44 (0)1344 29 6002                Ascot,
Email : <a class="txt-link txt-link-abbreviated" href="mailto:msmiles AT lucent 
DOT com">msmiles AT lucent DOT com</a>                  Berkshire, SL5 8AD

OK, who stopped payment on my reality check?
------------------------------------------------------------------
 
</pre></div>
</body>
</html>

--------------080005010405040809080709--




<Prev in Thread] Current Thread [Next in Thread>