Re: [ADSM-L] Restore of drives containing multi-millions of files

2014-09-15 12:38:47
Subject: Re: [ADSM-L] Restore of drives containing multi-millions of files
From: Roger Deschner <rogerd AT UIC DOT EDU>
Date: Mon, 15 Sep 2014 11:38:12 -0500
What is the OS platform of the client? It matters. The reason is that
the restore process must rebuild the directory tree on the client system
before it can repopulate it by restoring the files, and with that many
files it is likely a complex directory tree.

If it's Windows, you may be seeing a problem of restoring the directory
objects first. This can cause the same set of tapes to be mounted over
and over, each time restoring only a tiny amount of data. This age-old
ADSM and TSM problem is typically bypassed by using a separate
management class and DIRMC in the client options file. You can find a
lot of information about setting up DIRMC by searching the archives of
this list. Collocation can also help here. But, now that you've got the
problem, the only bypass is to use MOVE NODEDATA to a FILE storage pool
before restoring. I had to do this in order to restore a large Windows
client just last week.

Unix-family clients, including Linux and MacOSX, do not have this issue,
because the metadata for Unix directory objects fits into the TSM
Database itself, instead of being written into the storage pool
heirarchy and ending up on tape as happens with the larger Windows
directory objects.

Another issue you may be encountering is that, if the TSM client program
is 32-bit, it is unable to restore multi-millions of files at once due
to address space limits. "Memory efficient" settings seemed not to help,
because that forced the use of a large scratch disk area that it
accessed so heavily that it was taking days. We encountered this with a
32-bit Solaris client and 18,000,000 files, that ultimately had to be
restored in parts. 64-bit clients do not have this limitation.

Roger Deschner      University of Illinois at Chicago     rogerd AT uic DOT edu
======I have not lost my mind -- it is backed up on tape somewhere.=====

On Sun, 14 Sep 2014, Geoff Gill wrote:

>I was wondering if anyone has come across this in the past. I don't have 
>client side access but looks like they are at client level, which is 
>about all I can say right now.
>This client I'm told has over 14 million files on it. The restore which has 
>been running for some days has not completely stopped moving data but there 
>are 2 sessions that had tapes mounted moving data that are in a "Run" state 
>which show tapes mounted that no longer are. One of those actually dismounted 
>that tape 2 days ago, the other just yesterday. Another session is moving data 
>and the last is in a wait state for that tape.
>My question is what may be going on with the 2 that actually do NOT have the 
>tape mounted any longer but f=d shows they do.While I have seen customers with 
>clients like this in the past none have ever attempted such a large restore.
>Thank You
>Geoff Gill

<Prev in Thread] Current Thread [Next in Thread>