Re: [Networker] Long NDMP Backups

Hello,
400 Mb would be greatly limiting factor, did you mean 400 GB, I agree with 
limiting the savesets to match the filers performance and tune it 
gradually.
I would like the info on any TCP tuning that you have done, also, since 
DSA recovery are slow, could you not use a directly attached Tape drive 
for any restores ( atlest during DR).
Thanks



Matthew Huff <mhuff AT OX DOT COM> 
Sent by: EMC NetWorker discussion <NETWORKER AT LISTSERV.TEMPLE DOT EDU>
02/26/2008 03:46 PM
Please respond to
EMC NetWorker discussion <NETWORKER AT LISTSERV.TEMPLE DOT EDU>; Please respond 
to
Matthew Huff <mhuff AT OX DOT COM>


To
NETWORKER AT LISTSERV.TEMPLE DOT EDU
cc

Subject
Re: [Networker] Long NDMP Backups






I'm glad you are sure, because we are doing it right now and it's working 
well. Of course, we worked with Netapp engineering and this was their 
suggestion. They strongly suggest not having any saveset over 400MB, 
especially DAR restores can be extremely slow with very large savesets.

Obviously every filer is different. 10 may be too much, 5 may be just 
right, however doing everything in serial with just one saveset is going 
to be a major bottleneck. Tuning your network by using jumbo frames and 
making sure that the tcp sliding window is tuned is very important

BTW, I've been doing NDMP backups with Netapp since before Legato 
supported it, so I've got a bit of experience. We are currently backing up 
around 4TB. We are using about 6 savesets per filer. The full backups are 
taking around 6 hours, except for one saveset that is taking around 11 (we 
are currently migrating data around to break up the saveset). Once the 
migration is done, we should be back to around 6 hours. BTW, we have been 
forced to kick the backup off earlier than normal on a Friday (due to 
major power work being done over a weekend) and even with all the backups 
running, the system never had any issues even during the trading day.



----
Matthew Huff       | One Manhattanville Rd
OTA Management LLC | Purchase, NY 10577
www.otaotr.com     | Phone: 914-460-4039
aim: matthewbhuff  | Fax:   914-460-4139

-----Original Message-----
From: Yaron Zabary [mailto:yaron AT aristo.tau.ac DOT il] 
Sent: Tuesday, February 26, 2008 4:10 PM
To: EMC NetWorker discussion; Matthew Huff
Subject: Re: [Networker] Long NDMP Backups

Matthew Huff wrote:
> The main advantage is that it runs in parallel rather than in serial. 
For example, lets say your /vol/vol0 was 1TB, and had 10 qtrees each with 
100MB in it. You could increase the client parallelism in legato to 10, 
and when you started the backup with a saveset of:
> 
> /vol/vol0/dir_a
> /vol/vol0/dir_b
> /vol/vol0/dir_c
> /vol/vol0/dir_d
> /vol/vol0/dir_e
> /vol/vol0/dir_f
> /vol/vol0/dir_g
> /vol/vol0/dir_h
> /vol/vol0/dir_i
> /vol/vol0/dir_j
> 
> You would get 10 parallel backups each taking around 1/10 of what the 
volume backup would take. If you had the I/O and tape drive capacity, you 
would be reducing your backup time by 90%. Of course, that's an ideal 
situation.
> 

   I am quite sure that this is a great way of killing your filer. Our 
3050 can push at LTO-3 (~70MB/s) speed while consuming many CPU cycles 
(our CPU graphs are broken, so I cannot provide real numbers, but 20% 
seems about right). Considering this, running too many NDMP backups at 
once will make the filer unresponsive (assuming that it does any useful 
work, this might be unacceptable). It would not even get things to work 
any faster because if the filer is at 100% CPU utilization, it will 
become your bottleneck (it could even get you worse performance, as you 
will most likely have contention on your aggregate, volume or RAID group.

-- 

-- Yaron.

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type 
"signoff networker" in the body of the email. Please write to 
networker-request AT listserv.temple DOT edu if you have any problems with this 
list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER



To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the body of the email. Please write to 
networker-request AT listserv.temple DOT edu if you have any problems with this 
list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER