Networker

Re: [Networker] Long NDMP Backups

2008-02-26 16:49:25
Subject: Re: [Networker] Long NDMP Backups
From: Matthew Huff <mhuff AT OX DOT COM>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Date: Tue, 26 Feb 2008 16:46:41 -0500
I'm glad you are sure, because we are doing it right now and it's working well. 
Of course, we worked with Netapp engineering and this was their suggestion. 
They strongly suggest not having any saveset over 400MB, especially DAR 
restores can be extremely slow with very large savesets.

Obviously every filer is different. 10 may be too much, 5 may be just right, 
however doing everything in serial with just one saveset is going to be a major 
bottleneck. Tuning your network by using jumbo frames and making sure that the 
tcp sliding window is tuned is very important

BTW, I've been doing NDMP backups with Netapp since before Legato supported it, 
so I've got a bit of experience. We are currently backing up around 4TB. We are 
using about 6 savesets per filer. The full backups are taking around 6 hours, 
except for one saveset that is taking around 11 (we are currently migrating 
data around to break up the saveset). Once the migration is done, we should be 
back to around 6 hours. BTW, we have been forced to kick the backup off earlier 
than normal on a Friday (due to major power work being done over a weekend) and 
even with all the backups running, the system never had any issues even during 
the trading day.



----
Matthew Huff       | One Manhattanville Rd
OTA Management LLC | Purchase, NY 10577
www.otaotr.com     | Phone: 914-460-4039
aim: matthewbhuff  | Fax:   914-460-4139

-----Original Message-----
From: Yaron Zabary [mailto:yaron AT aristo.tau.ac DOT il] 
Sent: Tuesday, February 26, 2008 4:10 PM
To: EMC NetWorker discussion; Matthew Huff
Subject: Re: [Networker] Long NDMP Backups

Matthew Huff wrote:
> The main advantage is that it runs in parallel rather than in serial. For 
> example, lets say your /vol/vol0 was 1TB, and had 10 qtrees each with 100MB 
> in it. You could increase the client parallelism in legato to 10, and when 
> you started the backup with a saveset of:
>  
> /vol/vol0/dir_a
> /vol/vol0/dir_b
> /vol/vol0/dir_c
> /vol/vol0/dir_d
> /vol/vol0/dir_e
> /vol/vol0/dir_f
> /vol/vol0/dir_g
> /vol/vol0/dir_h
> /vol/vol0/dir_i
> /vol/vol0/dir_j
>  
> You would get 10 parallel backups each taking around 1/10 of what the volume 
> backup would take. If you had the I/O and tape drive capacity, you would be 
> reducing your backup time by 90%. Of course, that's an ideal situation.
> 

   I am quite sure that this is a great way of killing your filer. Our 
3050 can push at LTO-3 (~70MB/s) speed while consuming many CPU cycles 
(our CPU graphs are broken, so I cannot provide real numbers, but 20% 
seems about right). Considering this, running too many NDMP backups at 
once will make the filer unresponsive (assuming that it does any useful 
work, this might be unacceptable). It would not even get things to work 
any faster because if the filer is at 100% CPU utilization, it will 
become your bottleneck (it could even get you worse performance, as you 
will most likely have contention on your aggregate, volume or RAID group.

-- 

-- Yaron.

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the body of the email. Please write to 
networker-request AT listserv.temple DOT edu if you have any problems with this 
list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER