Thats the bit I am chasing, ie
Are they millions of small files? You will not get a steady strem if
this is the case?
If we are backing up blob data, ie a database, then I should expect a good
backup rate. If its small files, as you mention, then the rate will vary. Is
there any metric or way of determining the ratio I should be getting here ? Can
I see anywhere the data rate that the filer is presenting the data to be backed
up. I think this is the crux of the issue. If networker is simply backing up at
the rate that the data is being presented, then there is nothing I can do other
than stage to disk and then from there to tape to have a steady stream.
If I reduce the granularity of backups, ie the savesets, I assume this will
have no impact, as its the individual files that are still being streamed ?
Coty, Edward wrote:
>Are they millions of small files? You will not get a steady stream if
>this is the case?
>
>Check with STK about hardware compression?
>
>I am not sure you can compress NDMP data via networker. This would be
>done via a compression directive.
>
>What is you network? Are you using gigabit or fast Ethernet?
>
>What is the average speed you are seeing?
>
>EDWARD COTY
>OPEN SYSTEMS STORAGE ENGINEER, LCNA
>WORK - 973-533-2098
>CELL - 973-296-0918
>EDWARD.COTY AT AIG DOT COM
>
>-----Original Message-----
>From: bob flynn [mailto:bob.flynn AT s3group DOT com]
>Sent: Tuesday, July 18, 2006 8:22 AM
>To: Coty, Edward
>Cc: Legato NetWorker discussion
>Subject: Re: [Networker] NDMP over network speed question
>
>Answers are;
>
>What kind of data are you backing up?
>
>Mixed, ie text files and input/output files to EDA tools.
>
>
>What kind of hardware are you using?
>
>Storagetek SL500 library, with HP LTO2 drives.
>
>Do you have hardware compression on?
>
>Where within legato do I confirm this ?
>
>How many NDMP streams are you sending to one tape drive?
>
>I have varied this, but I get the same performace cycle. So to test,
>have taken all parallelism out, and still see the same result.
>
>Currently I am pegging the ndmp streams on the cpu/network capacity I am
>seeing on the filer and backup server. So I am running a max of 4 ndmp
>streams, and 4 streams to tape. The streams to tape may be ndmp or
>standard backups. Streaming to tape does not appear to be the issue, as
>in I get what I consider normal data rates with non-ndmp backups.
>
>Coty, Edward wrote:
>
>
>
>>What kind of data are you backing up? I am backing up file share
>>data(word,excel,PST) directly to tape. Drives are local to NDMP filer.
>>
>>
>I
>
>
>>am only seeing on average 27MBs. I am using LTOII drives. What kind of
>>hardware are you using? Do you have hardware compression on? How many
>>NDMP streams are you sending to one tape drive? The performance can
>>
>>
>just
>
>
>>depend on the type of data and the network you are running on?
>>
>>EDWARD COTY
>>OPEN SYSTEMS STORAGE ENGINEER, LCNA
>>WORK - 973-533-2098
>>CELL - 973-296-0918
>>EDWARD.COTY AT AIG DOT COM
>>-----Original Message-----
>>From: Legato NetWorker discussion
>>
>>
>[mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU]
>
>
>>On Behalf Of bob flynn
>>Sent: Tuesday, July 18, 2006 3:46 AM
>>To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
>>Subject: [Networker] NDMP over network speed question
>>
>>Hi,
>>
>>I have the following configuration
>>
>> * Networker Server 7.2.2
>> * SCSI attached LTO2 drives
>> * Linux RH 3 AS
>>
>>Netapp filers running NDMP over network, streaming data to backup
>>server.
>>
>>My question is to clarify my understanding of expected speeds of NDMP
>>backups.
>>
>>What I think happens is;
>>
>> * Networker server requests an ndmp backup from filer.
>> * Filer commences ndmp backup.
>> * ndmp backup runs through the ndmp phases. Once a tree walk of the
>> data is complete, and the file indexes have been created, it
>> starts streaming data to the backup server.
>>
>>My question is;
>>
>>I expect the job to work as follows;
>>
>> * First phases, no data streamed
>> * Once index is created, stream this data across
>> * Once data is being streamed, a solid stream of data from the
>>
>>
>filer
>
>
>> to backup server.
>>
>>What I see is;
>>
>> * First phases, no data streamed
>> * Once index is created, a trickle of data
>> * Once data is being streamed, data seems to spurt, ie
>>fast/slow/fast.
>>
>>What characteristics of ndmp data streaming should I see. Can I run any
>>tests to verify expected v's actual data rates, to determine if I am
>>bottlenecking backups at;
>>
>> * Filer ( CPU )
>> * Filer ( Network )
>> * Network
>> * Backup Server ( CPU )
>> * Backup Server ( network )
>> * Backup Server ( tapedrive )
>>
>>I am currently monitoring the following;
>>
>> * Filer ( CPU )
>> * Filer ( NIC )
>> * Network
>> * Backup Server ( CPU )
>> * Backup Server ( NIC )
>>
>>and am not seeing bottlenecks, but still seeing erratic backup
>>performance.
>>
>>Any hints to either understand this further, and baseline performance
>>are appreciated.
>>
>>-Bob
>>
>>
>>
>>
>>
>>
>>
>>The information contained in this e-mail and in any attachments is
>>confidential and is designated solely for the attention of the intended
>>recipient(s). If you are not an intended recipient, you must not use,
>>disclose, copy, distribute or retain this e-mail or any part thereof.
>>
>>
>If
>
>
>>you have received this e-mail in error, please notify the sender by
>>return e-mail and delete all copies of this e-mail from your computer
>>system(s).
>>Please direct any additional queries to: communications AT s3group DOT com.
>>Thank You.
>>
>>To sign off this list, send email to listserv AT listserv.temple DOT edu and
>>type "signoff networker" in the
>>body of the email. Please write to
>>
>>
>networker-request AT listserv.temple DOT edu
>
>
>>if you have any problems
>>wit this list. You can access the archives at
>>http://listserv.temple.edu/archives/networker.html or
>>via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
>>
>>
>>
>>
>
>
>The information contained in this e-mail and in any attachments is
>confidential and is designated solely for the attention of the intended
>recipient(s). If you are not an intended recipient, you must not use,
>disclose, copy, distribute or retain this e-mail or any part thereof. If
>you have received this e-mail in error, please notify the sender by
>return e-mail and delete all copies of this e-mail from your computer
>system(s).
>Please direct any additional queries to: communications AT s3group DOT com.
>Thank You.
>
>
The information contained in this e-mail and in any attachments is confidential
and is designated solely for the attention of the intended recipient(s). If you
are not an intended recipient, you must not use, disclose, copy, distribute or
retain this e-mail or any part thereof. If you have received this e-mail in
error, please notify the sender by return e-mail and delete all copies of this
e-mail from your computer system(s).
Please direct any additional queries to: communications AT s3group DOT com.
Thank You.
To sign off this list, send email to listserv AT listserv.temple DOT edu and
type "signoff networker" in the
body of the email. Please write to networker-request AT listserv.temple DOT edu
if you have any problems
wit this list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
|
