Veritas-bu

Re: [Veritas-bu] Max Fragment Size for Disk and LTO4 Based Storage Units

2008-02-01 15:29:10
Subject: Re: [Veritas-bu] Max Fragment Size for Disk and LTO4 Based Storage Units
From: Bryan Bahnmiller <bryan.bahnmiller AT managedmail DOT com>
Date: Fri, 01 Feb 2008 14:02:56 -0600
Ed,

> On Feb 1, 2008 9:11 AM, Bryan Bahnmiller <bryan.bahnmiller AT managedmail DOT 
> com>
> wrote:
> 
>>   I did a lot of testing at one point trying to tune NetBackup for
>> DSSU's, LTO2 and LTO3.
>>
>>   I found that 1024 MB fragments would never even spin up the LTO3
>> drive. It would basically start to speed up, then it would slow down
>> because it had to position for the next fragment. Total time - 6 seconds!
>>
>>   After much experimentation, I found that 6GB fragments worked best
>> for LTO2 and 10GB fragments best for LTO3. (Local FC attached high speed
>> disk to FC attached tape drives.) LTO4's could probably even use larger
>> fragments, I never tested them.
> 
> 
> Hi Bryan,
> 
> How did you determine that 10GB fragments were optimum?  Out of curiosity,
> why not 20?  We picked 20 as a rough number and not because of any serious
> benchmarking.  Did you just discover that it wouldn't write any faster than
> if you picked 10 so you stuck with that?

   I guessed. I tried, within reason, almost everything. I was using 
multiples of 2 and that didn't seem to make a difference. Then I just 
started picking numbers. 1, 2, 3, 4, 5, 6,,, I even tried some things 
like 10.5 and such. It was really strange. If you tried to plot it, it 
would probably be a fairly steady linear increase until I hit the peak 
number. Then it would bounce around a bit when I increased further, 
down, back up...

   Yep, you're right. I kind of hit the peak at 10, so I stuck with 
that. It might be one of those things that if you really got into it, 
you could probably start tweaking kernel parameters related to 
fibrechannel and SCSI and you might get a bit more, but I didn't have 
that kind of time.

> What sort of destaging performance are you getting?  We've found, in
> general, destaging performance is pretty poor but we've never been able to
> identify the bottlenecks and all the help we've gotten from NetBackup
> support and engineering hasn't made it any better.  We can write fast but we
> can't destage fast.

   The destaging performance never approached what I saw with backing up 
disk to tape. It seemed to help if you had the same fragment sizes with 
tape and disk, but if I recall correctly, it was in the range of 60% - 
70% the backup performance. Also, we were doing 2 tape copies in 
production - 1 for local recovery and 1 for DR. That slowed things down 
even further. For some reason when you do 2 copies, the synchronization 
hits you for about a 20% decrease in performance too.

   BTW, all of the testing was done with IBM p550's, AIX 5.3 and 
NetBackup 6.0 MP1.

   Another interesting side note I learned from the testing was that 
once you start burying files down several directory levels, your disk 
read performance starts to really stink. So using the staging 
directories with big files at the top of your directory hierarchy is the 
way to go.

> Thanks,
>    .../Ed
> 

   Bryan
_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu