Networker

Re: [Networker] How many RAID 5's should be used with DBO.

2006-01-04 19:28:17
Subject: Re: [Networker] How many RAID 5's should be used with DBO.
From: Peter Viertel <Peter.Viertel AT MACQUARIE DOT COM>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Date: Thu, 5 Jan 2006 11:26:03 +1100
That's what the text book says.... 

In reality, after a few weeks your filesystem will be somewhat
fragmented, and UFS spreads data all over the place, and the IO looks
pretty random for both reading and writing... so RAID3 ends up being no
better than RAID5....

I've been using DBO for about a year now.... Clariion 300GB disks in
5-way RAID3 - which gives a LUN size of about 1.2TB..

Currently I use solaris 9 SVM to stripe 2 LUNs into 2.5TB ufs
filesystems... The goal is generally to balance the write-load between
the two clariion service processors. I have been using about 3 or 4
adv_file devices at once.

If you use multiple filesystems for DBO the biggest problem is that
networker can at best only round-robin savesets between them (I use a
device target sessions of 1 for each adv_file device)....  This can
become a problem if you have a large number of *big* savesets... Eg
bigger than about 100GB... Because it is a lottery as to which device
the big ones get allocated to you will one day find that one device
shoots up to 100% and you are stuck until some staff is staged off it,
and that can lead to missing your service levels...

You asked about the UFS inodes... with solaris 9 when you create a UFS
filesystem greater than 1TB it will automatically give you the minimum
number of inodes which is about a million... Adv_file will only use a
couple of thousand at best, but you cant reduce the inode count any
further...

If you consider creating a huge filesystem.... Eg stripe or concat all
your LUNs together, and put all your DBO devices on this filesystem as
subdirectories.... It does work, but with UFS you would be in serious
trouble if you had to do a fsck....and of course you would use UFS
logging, and that would concentrate all the DBO device's write latency
to the region which is handling logging...

So... UFS is not cool, either way. And vxfs is just as bad, or worse....
What I am looking at now is Sun's SAM-FS.. With it you create a device
which is made out of a pool of LUNs for storing data, and another pool
for storing metadata...

So you can create some small high performance devices... Using RAID0+1
(striped/mirrored) (or RAID1+0 depending on religion)....  Use these for
metadata (and logging I believe)...   Create the rest as RAID5 or RAID3
and assign as data devices, SAM-FS manages how the data is spread
between the data devices, and is configurable... I am still trying to
see if you can 'ice' a device temporarily eg when you've lost a disk and
you want to minimise writing while reconstruction takes place...

Mount this device up as one enormous filesystem (or one per Pool)....
Create subdirectories for each DBA device... How many adv_file devices
you have depends on how many tape drives you have and your networker
license, but I would recommend having one more tape drive device than
adv_file devices so that you can stage to tape from all adv_files at
once but still have one drive free for restores....

Using SAM-FS like this is not too expensive.... The sun licence for
SAM-FS is per server   (note that you don't want to pay for the SAM-QFS
licenses which allow shared access and archiving and cost a LOT of money
and are based on number of terabytes).




-----Original Message-----
From: Legato NetWorker discussion [mailto:NETWORKER AT listserv.temple DOT edu]
On Behalf Of Itzik Meirson
Sent: Thursday, 5 January 2006 5:22 AM
To: NETWORKER AT listserv.temple DOT edu
Subject: Re: [Networker] How many RAID 5's should be used with DBO.

When doing backup to disk, EMC recommends, if using Clariion, to use
Raid 3 which is more suitable for the sequential type of access that a
filesysem used for backups will face.
Itzik 

-----Original Message-----
From: Legato NetWorker discussion [mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU]
On Behalf Of Michael Hurst
Sent: Wednesday, January 04, 2006 19:07
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Subject: [Networker] How many RAID 5's should be used with DBO.

Hi All,

I am in the process of adding a new 16TB disk array (32 * 500GB SATA
drives) to our NetWorker Server.
I am going to split the array into 2TB chucks and implement Legato's
DBO.
I am using 2TB chucks because of the cost of licensing additional
devices.
I am going to configure the array disks into a RAID 5 solution.
Since we have limited funds there is no test environment for me to work
with.
Therefore, I am going live to production and was hoping someone could
share their experiences.

Is there any advantage to running one big 16TB (raw) RAID 5 or having
several RAID 5 groups (possible 1 per 2TB partition)?
I know there will be more space loss with the extra RAID groups but will
this be offset with other factors?

Also, the backup server is running on Solaris.
Is it advisable to reduce the number of inodes when running newfs?
The default seems to be over kill for the number of NetWorker backup
streams.

Thanks in advance.

Cheers,
Michael Hurst
University of Toronto
Computing and Networking Services
EMAIL michael.hurst AT utoronto DOT ca

To sign off this list, send email to listserv AT listserv.temple DOT edu and
type "signoff networker" in the body of the email. Please write to
networker-request AT listserv.temple DOT edu if you have any problems wit this
list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or via RSS at
http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

To sign off this list, send email to listserv AT listserv.temple DOT edu and
type "signoff networker" in the body of the email. Please write to
networker-request AT listserv.temple DOT edu if you have any problems wit this
list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or via RSS at
http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER



NOTICE
This e-mail and any attachments are confidential and may contain copyright 
material of Macquarie Bank or third parties. If you are not the intended 
recipient of this email you should not read, print, re-transmit, store or act 
in reliance on this e-mail or any attachments, and should destroy all copies of 
them. Macquarie Bank does not guarantee the integrity of any emails or any 
attached files. The views or opinions expressed are the author's own and may 
not reflect the views or opinions of Macquarie Bank.

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the
body of the email. Please write to networker-request AT listserv.temple DOT edu 
if you have any problems
wit this list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER