Re: [Networker] Performance Tuning backup

Last question for BOB before I leave...

Bob, since you work at Chevron... 

Can you lower the price of oil?

Before you ask, no I can not give you a hot stock tip, but I can give
you the phone number of someone who can. <grin>

Have a good night folks! /ALE


-----Original Message-----
From: EMC NetWorker discussion [mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU] On
Behalf Of Wood, R A (Bob)
Sent: Monday, July 30, 2007 2:08 PM
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Subject: Re: [Networker] Performance Tuning backup


Curtis, 
        Your reasoning seems to assume that the problem lies upstream of
the backup server (which, I agree, is probably the case most of the
time). By using bigasm you are, effectively, giving the all clear to the
components downstream of the Networker server.

        In these days of virtualised devices and growing use of fabrics
(or other, novel, ways) to connect devices to backup server there's an
increasing possibility that bottlenecks could occur downstream of the
backup server. I'd be reluctant to rule out anything that helps pinpoint
possible pain areas

Bob 


 

>-----Original Message-----
>From: EMC NetWorker discussion 
>[mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU] On Behalf Of Curtis Preston
>Sent: 27 July 2007 10:29
>To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
>Subject: Re: [Networker] Performance Tuning backup
>
>Appreciate the respect, and I always love a good discussion.  
>Don't think that cause I'm Mr. Backup I don't want you to call 
>"BS!" when you feel it necessary.  If you think I'm full of 
>it, call it out!  One of us may change our position, or we may 
>find out that both ideas are good, and everyone who reads will 
>learn in the process.
>
>The main purpose of my post was to explain that if you're not 
>streaming one drive, there's nothing to be gained from taking 
>your backups and sending them to two drives.  Number one rule 
>in backup design, AFAIC.
>Whatever is causing things to hover at 30 MB/s, you're not 
>going to make it better by dividing the 30 MB/s among two 
>drives that each want 45 MB/s.  If anything, your 30 MB/s will 
>drop to 20-25 because you've now made the shoeshining worse.
>
>As to your idea on bigasm, I think that this one is just 
>different ways to skin the cat.  I've never gotten a lot from 
>tests that do things from memory.  I'd say the purpose of a 
>memory-based test is to rule out things (see, the tape drive 
>can go faster!), but they don't rule much out for me.  They 
>don't prove the actual effective throughput of the drives, 
>cause the compression ratio will be different.  If the data 
>you're sending from memory is either much more or much less 
>compressable than the real data, then your numbers will be 
>wrong. I've also seen such tests make the network look like 
>it's much more capable than it is.
>
>If I really don't know what's causing the problem, I try to 
>take one piece out of the puzzle at a time and see what 
>happens.  Remove the tape from the picture by backing up to 
>disk.  Hmm.  We hit the same numbers.
>Remove the network by backing up to a file type device on the client.
>Hmm.  Didn't get faster.  Not the network.  Must be the database app.
>
>(Something like that.)
>
>---
>W. Curtis Preston
>Backup Blog @ www.backupcentral.com
>VP Data Protection, GlassHouse Technologies 
>
>-----Original Message-----
>From: EMC NetWorker discussion 
>[mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU] On Behalf Of Howard Martin
>Sent: Friday, July 27, 2007 5:23 AM
>To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
>Subject: Re: [Networker] Performance Tuning backup
>
>With respect that is the purpose of using the bigasm test ie. 
>to find out what the server can send to the tapes without 
>involving networks or disks or the client, if that is ok then 
>you run it from the client to see if the network/client cpu if 
>ok. If that is alright then the problem is with the data or 
>disks on the client and throughput might be improved by tweaking
>
>target session and parallelism. 
>In my opinion it is far quicker to run a couple of bigasm test 
>to identify where the bottleneck is than to run a series of 
>tests aas suggested below.
>
>On Thu, 26 Jul 2007 18:39:53 -0400, Curtis Preston 
><cpreston AT GLASSHOUSE DOT COM> wrote:
>
>>I think you're going the wrong way.  And I disagree that the tape
>drives
>>aren't the bottleneck.  If they're LTO-2s and you're sending 
>them 10-15 
>>MB/s, they ARE.  They're shoeshining like crazy.  LTO-2s want 30-35
>MB/s
>>times compression.  I'll assume 1.5:1, which is probably reasonable.
>So
>>we're looking at a target number of 45-50 MB/s (or higher if you a 
>>higher compression ratio.)
>>
>>Try these backup settings.  Each time you change the setting, try 
>>backing up for a while and see what you get.  DON'T RUN ANY OTHER 
>>BACKUPS while you're doing this.  You need all other things constant:
>>
>>1. Set target sessions (TS) to one and client parallelism (CP) to one.
>>3. Set TS to 2 and CP to 2.
>>2. Set TS to 3 and CP to 3.
>>3. Set TS to 4 and CP to 4.
>>4. Set TS to 5 and CP to 5.
>>5. Set TS to 6 and CP to 6.
>>3. Set TS to 7 and CP to 7.
>>And so on, and so on, until you get to 40-45 MB/s on ONE 
>DRIVE or hit a 
>>point of decreasing marginal returns (you add another stream and tput 
>>doesn't get any better).
>>
>>If you can't get to 40-45 MB/s on one LTO-2 drive (assuming you're 
>>getting 1.5:1 compression or more), you're not going to get there by 
>>adding a second drive.  You're being limited on throughput somewhere, 
>>and splitting that limited throughput between two tape drives 
>will make 
>>it worse not better.
>>
>>If you do get to 40-45 MB/s (and I don't think you will), then see if 
>>you can get more.  Take the number of target sessions on one 
>drive to n 
>>(the highest number you had to go to above) and the other drive this
>>way:
>>
>>1. Set TS to n & 1 and CP to 4.
>>2. Set TS to n & 2 and CP to 6.
>>3. Set TS to n & 3 and CP to 8.
>>3. Set TS to n & 4 and CP to 10.
>>3. Set TS to n & 5 and CP to 12.
>>And so on, until you hit a point of decreasing marginal returns.
>>
>>Potential hold ups:
>>1. tput of the the disk that notes is on  (all RAID arrays not created
>>equal)
>>2. tput of the gbE interface on the client 3. tput of the gbe 
>interface 
>>on the master 4. tput of notes backup client 5. tput of the disk the 
>>client indices are on (probably not as this is notes and it doesn't 
>>require a lot of client index activity)
>>
>>
>>
>>
>>---
>>W. Curtis Preston
>>Backup Blog @ www.backupcentral.com
>>VP Data Protection, GlassHouse Technologies
>>
>>-----Original Message-----
>>From: EMC NetWorker discussion [mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU]
>On
>>Behalf Of E Gold
>>Sent: Thursday, July 26, 2007 12:42 PM
>>To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
>>Subject: Re: [Networker] Performance Tuning backup
>>
>>OK guys, as a test I changed my tape drive target sessions to 6 for
>each
>>of
>>my 6 tape drives.
>>
>>I then changed the parallelism on the client to 12 (it used to be 8)
>>
>>I then noticed my server will only go up to 32 sessions for
>parallelism,
>>given my max is 32 are the above settings OK?
>>
>>Also why am I limited to 32 max?
>>
>>Ill see tonight if I get more speed.
>>
>>thanks
>>
>>
>>____________________________________
>>This e-mail message is for the sole use of the intended recipient(s)
>and
>>may contain proprietary, confidential and/or privileged information.
>Any
>>unauthorized review, use, disclosure or distribution is 
>prohibited.  If 
>>you are not the intended recipient (or an employee or agent 
>responsible 
>>to deliver it to the intended recipient), you may not copy or deliver 
>>this message to anyone. In such case, you should destroy this message 
>>and kindly notify the sender by reply e-mail.
>>
>>To sign off this list, send email to listserv AT listserv.temple DOT edu and 
>>type "signoff networker" in the body of the email. Please write to 
>>networker-request AT listserv.temple DOT edu if you have any problems with
>this
>>list. You can access the archives at
>>http://listserv.temple.edu/archives/networker.html or via RSS at 
>>http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
>>
>>To sign off this list, send email to listserv AT listserv.temple DOT edu and
>type "signoff networker" in the body of the email. Please 
>write to networker-request AT listserv.temple DOT edu if you have any 
>problems with this
>
>list. You can access the archives at
>http://listserv.temple.edu/archives/networker.html or
>>via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
>
>To sign off this list, send email to listserv AT listserv.temple DOT edu and
>type "signoff networker" in the body of the email. Please write to
>networker-request AT listserv.temple DOT edu if you have any problems 
>with this
>list. You can access the archives at
>http://listserv.temple.edu/archives/networker.html or
>via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
>
>To sign off this list, send email to 
>listserv AT listserv.temple DOT edu and type "signoff networker" in 
>the body of the email. Please write to 
>networker-request AT listserv.temple DOT edu if you have any problems 
>with this list. You can access the archives at 
>http://listserv.temple.edu/archives/networker.html or
>via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
>

To sign off this list, send email to listserv AT listserv.temple DOT edu and
type "signoff networker" in the body of the email. Please write to
networker-request AT listserv.temple DOT edu if you have any problems with this
list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
--------------------------------------------------------

This message w/attachments (message) may be privileged, confidential or 
proprietary, and if you are not an intended recipient, please notify the 
sender, do not use or share it and delete it. Unless specifically indicated, 
this message is not an offer to sell or a solicitation of any investment 
products or other financial product or service, an official confirmation of any 
transaction, or an official statement of Merrill Lynch. Subject to applicable 
law, Merrill Lynch may monitor, review and retain e-communications (EC) 
traveling through its networks/systems. The laws of the country of each 
sender/recipient may impact the handling of EC, and EC may be archived, 
supervised and produced in countries other than the country in which you are 
located. This message cannot be guaranteed to be secure or error-free. This 
message is subject to terms available at the following link: 
http://www.ml.com/e-communications_terms/. By messaging with Merrill Lynch you 
consent to the foregoing.
--------------------------------------------------------

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the body of the email. Please write to 
networker-request AT listserv.temple DOT edu if you have any problems with this 
list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER