Veritas-bu

[Veritas-bu] Streaming and large data sets

2004-03-19 13:10:33
Subject: [Veritas-bu] Streaming and large data sets
From: gary.andresen AT pnwdata DOT com (Gary Andresen)
Date: Fri, 19 Mar 2004 10:10:33 -0800
Multiple streams from one device are only bad if the device is a SINGLE
drive or a drive that is just mirrored. Multiple streams from said device
will cause quite a lot of head thrashing as the heads have to keep jumping
from spot to spot and your performance will suffer.

You should be able to find the sweet spot for how many streams you can pump
to a single AIT drive (multiplexing) and for how many streams you can pump
from your RAID (performance will start to drop off). 

In your situation running 4 streams with 1 stream per drive should work fine
(I assume that the file system and tape drives are on the same system or
your using GigE for your network interface. 100Mbit Ethernet is limited to
about 32Gbyte/hr.)


Gary Andresen
Impossible Happens, Plan Ahead
Pacific Northwest Data Inc.
Tel: 503.701.5185
Fax: 503.692.3910
gary.andresen AT pnwdata DOT com
www.pnwdata.com

> -----Original Message-----
> From: veritas-bu-admin AT mailman.eng.auburn DOT edu [mailto:veritas-bu-
> admin AT mailman.eng.auburn DOT edu] On Behalf Of Justin C. Lloyd
> Sent: Friday, March 19, 2004 9:28 AM
> To: veritas-bu AT mailman.eng.auburn DOT edu
> Subject: [Veritas-bu] Streaming and large data sets
> 
> Ok, maybe I'm missing something.
> 
> One recommendation regarding streaming is not to have multiple streams
> off the same physical device.  By physical device, I am also taking that
> to mean a RAID device that spans multiple disks and/or disk arrays.  In
> my case I have four arrays partitioned into 11 LUNs each and then my
> filesystem is built on a volume striped across all 44 LUNs.
> 
> So, I have a 400 GB (for now, but growing) data set on that filesystem.
>   My library's 8 AIT-3 tape drives can write at up to 40 GB/h each.
> Therefore if I were to have a single stream the data set would take 10
> hours to back up.  But if I want it to complete in, say, 2 hours, I'd
> have to use 4 drives and hence define 4 roughly equal size streams
> (which is what I currently do), which I occasionally have to rebalance
> as the data set grows.  But this streaming goes against the above
> recommendation.
> 
> It's possible this really isn't a problem if it just boils down to the
> drawback of the streaming taking more time due to additional head
> movement.  Breaking up a huge stream more than compensates for that
> overhead.
> 
> Any comments or suggestions?
> 
> Justin
> 
> --
> Justin C. Lloyd
> Unix System Administrator
> MCI System Technology Solutions
> Office 703.886.3219 Vnet 806.3219
> 
> _______________________________________________
> Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu




<Prev in Thread] Current Thread [Next in Thread>