Veritas-bu

[Veritas-bu] backing up LARGE oracle databases (looking for s uggestions)

2005-01-28 20:36:30
Subject: [Veritas-bu] backing up LARGE oracle databases (looking for s uggestions)
From: Dwayne.Brzozowski AT mail.va DOT gov (Brzozowski, Dwayne)
Date: Fri, 28 Jan 2005 19:36:30 -0600
This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

------_=_NextPart_001_01C505A2.F4FD1156
Content-Type: text/plain;
        charset="iso-8859-1"

How are the Oracle backups. I had the same problem you are experiencing. I
had the backups for one customer who had a 1 TB database, with the files set
to /u01,/02, etc. That was a nightmare trying to back up that amount of data
in the window we were given. With that structure, backups would constantly
fail because a new database was added to a mount point /U02/* and the backup
group were never notified. This customer had 5 domains spread across a Sun
E10K and the Oracle instances spanned across 3 of these domains, which made
things more difficult. To fix this problem and it work extremely well, one
of our backup admins, Blaine-who is on this list, removed all the /U01,/U02
files sets, and created policies for each instance. So, one policy would
have one, two, or all 3 domains/servers, with only one oracle instance. This
gave the admins control over what was backed up and forced the dba's to
notify us when a new database was being created. With this amount of data,
we wrote to 4-9940A drives with a back window of  2230-0430 and rarely had a
failure. With the restructuring I described above, you can piggy back
databases with many small files with binaries to increase throughput. This
was only one customer and we backed up 4-6 TB's every night. Let me know if
this is something you want to try and I'll can give  you more specific
information on this. 
 
 
-Dwayne
Dwayne J. Brzozowski
Department of Veterans Affairs
Night Shift Supervisor-Unix Group
Austin Automation center
(512)326-6728 work
dwayne.brzozowski AT mail.va DOT gov


-----Original Message-----
From: veritas-bu-admin AT mailman.eng.auburn DOT edu
[mailto:veritas-bu-admin AT mailman.eng.auburn DOT edu]On Behalf Of Dobbertien,
Matthew
Sent: Thursday, January 27, 2005 4:59 PM
To: veritas-bu AT mailman.eng.auburn DOT edu
Subject: RE: [Veritas-bu] backing up LARGE oracle databases (looking for
suggestions)



I don't necessarily have any good suggestions, but we have a similar issue
and are hoping to find some resolution.  We currently have a 600 GB database
that is expected to grow to over 2 TB in the near future.  We are trying to
plan for the additional capacity and make the users aware of timeframes for
backups.  Currently we are backing up over Gb Ethernet, which seems to max
out at about 60-70 MB/s, even in vendor (Sun) tests.  This is barely enough
to maximize throughput on 1 LTO2 drive, taking into account compression.  We
are currently writing to 2 LTO2 drives with an L700e library (8 total LTO2
drives, soon to be 10).  We also have a large VTL (about 50 TB), which is
not yet installed, which will serve as the primary location for backups.  

 

Right now, the chokepoint is the GigE connections.  I'd like to be able to
use snapshots to backup the data directly to tape, but there doesn't seem to
be a good way to manage the snaphshot process using Netbackup with Clariion
arrays.  We also considered making the servers SAN media servers, but since
these applications reside on a Sun15k, the cost is prohibitive, and the
extra complexity of having to involve the system owners for backup
maintenance is not desired.  

 

Any comments on expected backup speed, methods of improving backup speed for
large backups (I understand that the world record is 2 TB an hour, but I
don't think that we can achieve that in our environment) and any other
information would be helpful.  If you could convince the database owners to
use RMAN, that would help as well, but I'm not going to address that now.

 

Matthew Dobbertien
Sr. Systems Analyst
Chicago Tribune




  _____  


From: veritas-bu-admin AT mailman.eng.auburn DOT edu
[mailto:veritas-bu-admin AT mailman.eng.auburn DOT edu] On Behalf Of Rob Haig
Sent: Thursday, January 27, 2005 10:14 AM
To: veritas-bu AT mailman.eng.auburn DOT edu
Subject: [Veritas-bu] backing up LARGE oracle databases (looking for
suggestions)

 

So I'm trying to wrap my head around this problem.  I'm backing up a 4TB
oracle database with 4 LTO2 drives, connected to the server via 1GB
Ethernet.  That's not that bad of a problem.  RMAN, lvl0 backups twice a
week, plus daily lvl1 backups plus archive logs.  However, when you throw
into the mix that I have 20 of these servers (4TB is the average DB size,
they very from about 200G to 7TB) to back up, it gets more interesting.
I'll be able to spend money eventually, but not right now.  My eventual plan
is a few fully populated SL500 libraries with 4 drives each.  In the final
plan, I'd like to have 8 drives each library and about 150TB of SAN in
between the DB servers and tape drives with serverless backup from the SAN
to tape.  Or, a big honkin' virtual tape library full of disks that can do
serverless backup and has several SAN connections.  

 

But right now, I'm limited to 2, maybe 3, 2-drive LTO2 changers with enough
slots to back up about 8TB of data each before I have to shuffle out full
tapes.  I've been pounding my head on this problem for a few days now and
every time I think I have it sorted out so it's not that bad, I remember
that I have 20 of these databases.

 

Anyone have any suggestions?

 

-- 

Rob


------_=_NextPart_001_01C505A2.F4FD1156
Content-Type: text/html;
        charset="iso-8859-1"

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML xmlns="http://www.w3.org/TR/REC-html40"; xmlns:v = 
"urn:schemas-microsoft-com:vml" xmlns:o = 
"urn:schemas-microsoft-com:office:office" xmlns:w = 
"urn:schemas-microsoft-com:office:word" xmlns:st1 = 
"urn:schemas-microsoft-com:office:smarttags"><HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">


<META content="MSHTML 6.00.2800.1480" name=GENERATOR><!--[if !mso]>
<STYLE>
v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</STYLE>
<![endif]--><o:SmartTagType 
downloadurl="http://www.5iamas-microsoft-com:office:smarttags"; name="City" 
namespaceuri="urn:schemas-microsoft-com:office:smarttags"></o:SmartTagType><o:SmartTagType
 
downloadurl="http://www.5iantlavalamp.com/"; name="place" 
namespaceuri="urn:schemas-microsoft-com:office:smarttags"></o:SmartTagType><!--[if
 !mso]>
<STYLE>
st1\:*{behavior:url(#default#ieooui) }
</STYLE>
<![endif]-->
<STYLE>
<!--
 /* Font Definitions */
 @font-face
        {font-family:Tahoma;
        panose-1:2 11 6 4 3 5 4 4 2 4;}
 /* Style Definitions */
 p.MsoNormal, li.MsoNormal, div.MsoNormal
        {margin:0in;
        margin-bottom:.0001pt;
        font-size:12.0pt;
        font-family:"Times New Roman";}
a:link, span.MsoHyperlink
        {color:blue;
        text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
        {color:purple;
        text-decoration:underline;}
span.emailstyle17
        {font-family:Arial;
        color:windowtext;}
span.EmailStyle18
        {mso-style-type:personal-reply;
        font-family:Arial;
        color:navy;}
@page Section1
        {size:8.5in 11.0in;
        margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
        {page:Section1;}
-->
</STYLE>
</HEAD>
<BODY lang=EN-US vLink=purple link=blue>
<DIV><SPAN class=808052201-29012005><FONT face=Arial color=#0000ff size=2>How 
are the Oracle backups. I had the same problem you are experiencing. I had the 
backups for one customer who had a 1 TB database, with the files set to 
/u01,/02, etc. That was a nightmare trying to back up that amount of data in 
the 
window we were given. With that structure, backups would constantly fail 
because 
a new database was added to a mount point /U02/* and the backup group were 
never 
notified. This customer had 5 domains spread across a Sun E10K and the Oracle 
instances spanned across 3 of these domains, which made things more difficult. 
To fix this problem and it work extremely well, one of our backup admins, 
Blaine-who is on this list, removed all the /U01,/U02 files sets, and created 
policies for each instance. So, one policy would have one, two, or all 3 
domains/servers, with only one oracle instance. This gave the admins control 
over what was backed up and forced the dba's to notify us when a new database 
was being created. With this amount of data, we wrote to 4-9940A drives with a 
back window of&nbsp; 2230-0430 and rarely had a failure. With the restructuring 
I described above, you can piggy back databases with many small files with 
binaries to increase throughput. This was only one customer and we backed up 
4-6 
TB's every night. Let me know if this is something you want to try and I'll can 
give&nbsp; you more specific information on this. </FONT></SPAN></DIV>
<DIV><SPAN class=808052201-29012005><FONT face=Arial color=#0000ff 
size=2></FONT></SPAN>&nbsp;</DIV>
<DIV><SPAN class=808052201-29012005><FONT face=Arial color=#0000ff 
size=2></FONT></SPAN>&nbsp;</DIV>
<DIV><SPAN class=808052201-29012005><FONT face=Arial color=#0000ff 
size=2>-Dwayne</FONT></SPAN></DIV>
<DIV><SPAN class=808052201-29012005>
<P><FONT size=2>Dwayne J. Brzozowski<BR>Department of Veterans Affairs<BR>Night 
Shift Supervisor-Unix Group<BR>Austin Automation center<BR>(512)326-6728 
work<BR>dwayne.brzozowski AT mail.va DOT gov<BR></FONT></P></SPAN></DIV>
<BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px">
  <DIV class=OutlookMessageHeader dir=ltr align=left><FONT face=Tahoma 
  size=2>-----Original Message-----<BR><B>From:</B> 
  veritas-bu-admin AT mailman.eng.auburn DOT edu 
  [mailto:veritas-bu-admin AT mailman.eng.auburn DOT edu]<B>On Behalf Of 
  </B>Dobbertien, Matthew<BR><B>Sent:</B> Thursday, January 27, 2005 4:59 
  PM<BR><B>To:</B> veritas-bu AT mailman.eng.auburn DOT edu<BR><B>Subject:</B> 
RE: 
  [Veritas-bu] backing up LARGE oracle databases (looking for 
  suggestions)<BR><BR></FONT></DIV>
  <DIV class=Section1>
  <P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN 
  style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">I don't necessarily 
  have any good suggestions, but we have a similar issue and are hoping to find 
  some resolution.&nbsp; We currently have a 600 GB database that is expected 
to 
  grow to over 2 TB in the near future.&nbsp; We are trying to plan for the 
  additional capacity and make the users aware of timeframes for backups.&nbsp; 
  Currently we are backing up over Gb Ethernet, which seems to max out at about 
  60-70 MB/s, even in vendor (Sun) tests.&nbsp; This is barely enough to 
  maximize throughput on 1 LTO2 drive, taking into account compression.&nbsp; 
We 
  are currently writing to 2 LTO2 drives with an L700e library (8 total LTO2 
  drives, soon to be 10).&nbsp; We also have a large VTL (about 50 TB), which 
is 
  not yet installed, which will serve as the primary location for 
backups.&nbsp; 
  <o:p></o:p></SPAN></FONT></P>
  <P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN 
  style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: 
Arial"><o:p>&nbsp;</o:p></SPAN></FONT></P>
  <P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN 
  style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Right now, the 
  chokepoint is the GigE connections.&nbsp; I'd like to be able to use 
snapshots 
  to backup the data directly to tape, but there doesn't seem to be a good way 
  to manage the snaphshot process using Netbackup with Clariion arrays.&nbsp; 
We 
  also considered making the servers SAN media servers, but since these 
  applications reside on a Sun15k, the cost is prohibitive, and the extra 
  complexity of having to involve the system owners for backup maintenance is 
  not desired.&nbsp; <o:p></o:p></SPAN></FONT></P>
  <P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN 
  style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: 
Arial"><o:p>&nbsp;</o:p></SPAN></FONT></P>
  <P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN 
  style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Any comments on 
  expected backup speed, methods of improving backup speed for large backups (I 
  understand that the world record is 2 TB an hour, but I don't think that we 
  can achieve that in our environment) and any other information would be 
  helpful.&nbsp; If you could convince the database owners to use RMAN, that 
  would help as well, but I'm not going to address that 
  now.<o:p></o:p></SPAN></FONT></P>
  <P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN 
  style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: 
Arial"><o:p>&nbsp;</o:p></SPAN></FONT></P>
  <DIV>
  <P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN 
  style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Matthew 
  Dobbertien<BR>Sr. Systems Analyst<BR><st1:City w:st="on"><st1:place 
  w:st="on">Chicago</st1:place></st1:City> 
  Tribune<BR><BR></SPAN></FONT><o:p></o:p></P></DIV>
  <DIV>
  <DIV class=MsoNormal style="TEXT-ALIGN: center" align=center><FONT 
  face="Times New Roman" size=3><SPAN style="FONT-SIZE: 12pt">
  <HR tabIndex=-1 align=center width="100%" SIZE=2>
  </SPAN></FONT></DIV>
  <P class=MsoNormal><B><FONT face=Tahoma size=2><SPAN 
  style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: 
Tahoma">From:</SPAN></FONT></B><FONT 
  face=Tahoma size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Tahoma"> 
  veritas-bu-admin AT mailman.eng.auburn DOT edu 
  [mailto:veritas-bu-admin AT mailman.eng.auburn DOT edu] <B><SPAN 
  style="FONT-WEIGHT: bold">On Behalf Of </SPAN></B>Rob Haig<BR><B><SPAN 
  style="FONT-WEIGHT: bold">Sent:</SPAN></B> Thursday, January 27, 2005 10:14 
  AM<BR><B><SPAN style="FONT-WEIGHT: bold">To:</SPAN></B> 
  veritas-bu AT mailman.eng.auburn DOT edu<BR><B><SPAN 
  style="FONT-WEIGHT: bold">Subject:</SPAN></B> [Veritas-bu] backing up LARGE 
  oracle databases (looking for suggestions)</SPAN></FONT><o:p></o:p></P></DIV>
  <P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN 
  style="FONT-SIZE: 12pt"><o:p>&nbsp;</o:p></SPAN></FONT></P>
  <P class=MsoNormal><FONT face=Arial size=2><SPAN 
  style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">So I'm trying to wrap my head 
  around this problem.&nbsp; I'm backing up a 4TB oracle database with 4 LTO2 
  drives, connected to the server via 1GB Ethernet.&nbsp; That's not that bad 
of 
  a problem.&nbsp; RMAN, lvl0 backups twice a week, plus daily lvl1 backups 
plus 
  archive logs.&nbsp; However, when you throw into the mix that I have 20 of 
  these servers (4TB is the average DB size, they very from about 200G to 7TB) 
  to back up, it gets more interesting.&nbsp; I'll be able to spend money 
  eventually, but not right now.&nbsp; My eventual plan is a few fully 
populated 
  SL500 libraries with 4 drives each.&nbsp; In the final plan, I'd like to have 
  8 drives each library and about 150TB of SAN in between the DB servers and 
  tape drives with serverless backup from the SAN to tape. &nbsp;Or, a big 
  honkin' virtual tape library full of disks that can do serverless backup and 
  has several SAN connections.&nbsp; </SPAN></FONT><o:p></o:p></P>
  <P class=MsoNormal><FONT face=Arial size=2><SPAN 
  style="FONT-SIZE: 10pt; FONT-FAMILY: 
Arial">&nbsp;</SPAN></FONT><o:p></o:p></P>
  <P class=MsoNormal><FONT face=Arial size=2><SPAN 
  style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">But right now, I'm limited to 2, 
  maybe 3, 2-drive LTO2 changers with enough slots to back up about 8TB of data 
  each before I have to shuffle out full tapes.&nbsp; I've been pounding my 
head 
  on this problem for a few days now and every time I think I have it sorted 
out 
  so it's not that bad, I remember that I have 20 of these 
  databases.</SPAN></FONT><o:p></o:p></P>
  <P class=MsoNormal><FONT face=Arial size=2><SPAN 
  style="FONT-SIZE: 10pt; FONT-FAMILY: 
Arial">&nbsp;</SPAN></FONT><o:p></o:p></P>
  <P class=MsoNormal><FONT face=Arial size=2><SPAN 
  style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Anyone have any 
  suggestions?</SPAN></FONT><o:p></o:p></P>
  <P class=MsoNormal><FONT face=Arial size=2><SPAN 
  style="FONT-SIZE: 10pt; FONT-FAMILY: 
Arial">&nbsp;</SPAN></FONT><o:p></o:p></P>
  <P class=MsoNormal><FONT face=Arial size=2><SPAN 
  style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">-- </SPAN></FONT><o:p></o:p></P>
  <P class=MsoNormal><FONT face=Arial size=2><SPAN 
  style="FONT-SIZE: 10pt; FONT-FAMILY: 
Arial">Rob</SPAN></FONT><o:p></o:p></P></DIV></BLOCKQUOTE></BODY></HTML>

------_=_NextPart_001_01C505A2.F4FD1156--

<Prev in Thread] Current Thread [Next in Thread>
  • [Veritas-bu] backing up LARGE oracle databases (looking for s uggestions), Brzozowski, Dwayne <=