Networker

Re: [Networker] NDMP over network speed question

2006-07-19 09:17:11
Subject: Re: [Networker] NDMP over network speed question
From: Jeff Mery <jeff.mery AT NI DOT COM>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Date: Wed, 19 Jul 2006 08:16:41 -0500
How full are the volumes you're backing up?  We've noticed that as 
utilization approaches 85% - 90%, backup performance goes in the crapper. 
This is most likely due to fragmentation and ONTAP having to search for 
blocks rather than being able to just stream contiguous blocks.  Removing 
a significant portion of the data (down below 85%) had an immediate impact 
on backup speeds.  (We could probably use a wafl scan relocate too, but 
I'm not in the mood to delete all of our snapshots  =)).

How busy are the disks when your backups are running?  You can use statit 
from advanced mode to see exactly what each individual disk is doing. 
While sysstat may show fairly low utilization, you could have hot disks 
causing bottlenecks (too much data trying to come from too few spindles). 
This usually happens when volumes/aggregates are grown by one or two disks 
at a time rather than a larger number (we use 3 - 5 as a minimum).

Not sure if these are the true causes, but they're worth checking...

Jeff Mery - MCSE, MCP
National Instruments

-------------------------------------------------------------------------
"Allow me to extol the virtues of the Net Fairy, and of all the fantastic
dorks that make the nice packets go from here to there. Amen."
TB - Penny Arcade
-------------------------------------------------------------------------



bob flynn <bob.flynn AT S3GROUP DOT COM> 
Sent by: Legato NetWorker discussion <NETWORKER AT LISTSERV.TEMPLE DOT EDU>
07/19/2006 02:56 AM
Please respond to
Legato NetWorker discussion <NETWORKER AT LISTSERV.TEMPLE DOT EDU>; Please 
respond to
bob flynn <bob.flynn AT S3GROUP DOT COM>


To
NETWORKER AT LISTSERV.TEMPLE DOT EDU
cc

Subject
Re: [Networker] NDMP over network speed question






I was previously using this configuration, but it had the following
drawbacks;

- The drive(s) were direct attached to the netapp, and were not
available for other use, this was an inefficent use of resources. ( We
did not have a SAN configuration, to allow drives to be shared between
multiple devices ).
- It required that legato indexes still had to be streamed back to
legato server, as legato server has to see the indexes to allow browse
recovery.

Looking at the netapp nic stats, its a gigabit interface, which is clean
and peaking at approx 10/15%, similarly for the legato server ( 2 bonded
gigabit NICs ) so I do not think that the NIC is a bottleneck ?

-Bob

Robert Maiello wrote:

>So your doing NDMP over the network to drives attached to the backup
>server, they're not attached to the NetApp ?
>
>If so, I would think your top speeds may be limited your servers NIC(s).
>
>Robert Maiello
>Pioneer Data Systems
>
>
>On Tue, 18 Jul 2006 08:46:20 +0100, bob flynn <bob.flynn AT S3GROUP DOT COM> 
wrote:
>
> 
>
>>Hi,
>>
>>I have the following configuration
>>
>>   * Networker Server 7.2.2
>>   * SCSI attached LTO2 drives
>>   * Linux RH 3 AS
>>
>>Netapp filers running NDMP over network, streaming data to backup 
server.
>>
>>My question is to clarify my understanding of expected speeds of NDMP
>>backups.
>>
>>What I think happens is;
>>
>>   * Networker server requests an ndmp backup from filer.
>>   * Filer commences ndmp backup.
>>   * ndmp backup runs through the ndmp phases. Once a tree walk of the
>>     data is complete, and the file indexes have been created, it
>>     starts streaming data to the backup server.
>>
>>My question is;
>>
>>I expect the job to work as follows;
>>
>>   * First phases, no data streamed
>>   * Once index is created, stream this data across
>>   * Once data is being streamed, a solid stream of data from the filer
>>     to backup server.
>>
>>What I see is;
>>
>>   * First phases, no data streamed
>>   * Once index is created, a trickle of data
>>   * Once data is being streamed, data seems to spurt, ie 
fast/slow/fast.
>>
>>What characteristics of ndmp data streaming should I see. Can I run any
>>tests to verify expected v's actual data rates, to determine if I am
>>bottlenecking backups at;
>>
>>   * Filer ( CPU )
>>   * Filer ( Network )
>>   * Network
>>   * Backup Server ( CPU )
>>   * Backup Server ( network )
>>   * Backup Server ( tapedrive )
>>
>>I am currently monitoring the following;
>>
>>   * Filer ( CPU )
>>   * Filer ( NIC )
>>   * Network
>>   * Backup Server ( CPU )
>>   * Backup Server ( NIC )
>>
>>and am not seeing bottlenecks, but still seeing erratic backup 
performance.
>>
>>Any hints to either understand this further, and baseline performance
>>are appreciated.
>>
>>-Bob
>>
>>
>>
>>
>>
>>
>>
>>The information contained in this e-mail and in any attachments is 
>> 
>>
>confidential and is designated solely for the attention of the intended 
>recipient(s). If you are not an intended recipient, you must not use, 
>disclose, copy, distribute or retain this e-mail or any part thereof. If 
>you have received this e-mail in error, please notify the sender by 
return 
>e-mail and delete all copies of this e-mail from your computer system(s).
> 
>
>>Please direct any additional queries to: communications AT s3group DOT com.
>>Thank You.
>>
>>To sign off this list, send email to listserv AT listserv.temple DOT edu and 
>> 
>>
>type "signoff networker" in the
> 
>
>>body of the email. Please write to networker-request AT listserv.temple DOT 
>>edu 

>> 
>>
>if you have any problems
> 
>
>>wit this list. You can access the archives at 
>> 
>>
>http://listserv.temple.edu/archives/networker.html or
> 
>
>>via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
>>=========================================================================
>> 
>>
>
>
> 
>



The information contained in this e-mail and in any attachments is 
confidential and is designated solely for the attention of the intended 
recipient(s). If you are not an intended recipient, you must not use, 
disclose, copy, distribute or retain this e-mail or any part thereof. If 
you have received this e-mail in error, please notify the sender by return 
e-mail and delete all copies of this e-mail from your computer system(s).
Please direct any additional queries to: communications AT s3group DOT com.
Thank You.

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type 
"signoff networker" in the
body of the email. Please write to networker-request AT listserv.temple DOT edu 
if you have any problems
wit this list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER


To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the
body of the email. Please write to networker-request AT listserv.temple DOT edu 
if you have any problems
wit this list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER