Taking opportunity to mount soapbox on this issue:  

I have SO many customer that are facing this problem.
Vendors of giant NAS devices *should* be providing better, decent solutions to 
back them up.
It's very do-able, it's just that crappy vendors don't care what they dump on 
you.

NetApp, for example, has implemented their SnapDiff API that provides true 
incrementals, and it works.
I have no other reason to plug NetApp, I'm just pointing out that other vendors 
could do the same thing, if they wanted to - it's not impossible.

I know, in most cases backup folks have hardware dropped on us without any 
input into the purchase, and we are just stuck with it.
BUT the only thing that is going to solve this industry-wide problem in the 
long run, is whenever tech people hear there is a purchase going down, we step 
in and tell management STOP! WHAT ARE YOU THINKING?!?, and boycott vendors who 
inflict bad technology on us when they could do better.

Soapbox off.

W  


-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf Of 
Skylar Thompson
Sent: Tuesday, May 06, 2014 1:10 PM
To: ADSM-L AT VM.MARIST DOT EDU
Subject: Re: [ADSM-L] file system backups of a Dell NDMP Equallogic device

Agreed that NDMP is awful. I think you're on the right track with your NFS 
solution. Here's how we deal with it now:

* Break your NFS volumes up into different mount points (i.e. break
  /net/example/foo into /net/example/foo/dir1, /net/example/foo/dir2, ...).
  Reference each mount point using -domain.

* Break each schedule up using a combination of schedule nodes and proxy
  nodes. Associate the schedules with the schedule nodes, and then
  associate the storage back to the underlying node using -asnode. For
  instance, assuming a storage node of NFS-EXAMPLE, create two schedule
  nodes NFS-EXAMPLE-FOO-DIR1 and NFS-EXAMPLE-FOO-DIR2. Associate the first
  directory with the first node using "-domain=/net/example/foo/dir1" in the
  client options, and the second directory with the second node using
  "domain=/net/example/foo/dir2". Use -asnode=NFS-EXAMPLE in both.

This doesn't avoid all the concurrency problems; looking back, it might have 
been better just to use many nodes with storage directly associated with them 
and not use -asnode.

On Mon, May 05, 2014 at 07:26:36PM -0400, Dury, John C. wrote:
> Sorry  for revisiting this but I'm in a predicament now. Trying to backup the 
> NDMP device is a miserable failure and frankly just ugly.  I honestly can't 
> see why anyone would use TSM to backup any NDMP devices except for maybe 
> speed issues.
>
> We decided to mount all of the NFS shares locally on the TSM server and allow 
> them to be backed up that way but now the problem is that even with 
> resourceutilization set to 20, it still takes 18+ hours just to do an 
> incremental because there are millions and millions of files in all of those 
> NFS shares. So this isn't going to work either. I can try the proxy node 
> solution but frankly I'm skeptical about it also because of the tremendous 
> number of small files. Of course this is all for a mission critical 
> application so I have to come up with a workable solution and I'm running out 
> of ideas.

--
-- Skylar Thompson (skylar2 AT u.washington DOT edu)
-- Genome Sciences Department, System Administrator
-- Foege Building S046, (206)-685-7354
-- University of Washington School of Medicine