Networker

Re: [Networker] Indexes disappearing ..

2011-09-28 09:17:04
Subject: Re: [Networker] Indexes disappearing ..
From: Michael Leone <Michael.Leone AT PHA.PHILA DOT GOV>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Date: Wed, 28 Sep 2011 09:14:54 -0400
> Something must have changed. Maybe accidentally (a recent NW server 
crash, 

No.

> a change on the DNS...?)

No.

> If the saveset is saved but the index is removed, then you may be 
> experiencing 
> some client identity issue.
> 
> Have you tried running 
>   nsrim -X (once)
>   nsrck -L6 (3 times)

Yes.

> 
> (that didn't work for me ehan I had a similar problem but the followig 
> procedure produced some warnings about an old client name and worked for 
me 
> fine with just one client:
>  - remove both clients  (remove the index folders too)
> -  recreate both clients using the existing clientID's but with a 
resolvable 
> dummy  client name each (e.g. "dummy1" and "dummy2" -- you will need to 
> create two fake entries  on the /etc/hosts *before* you attempt to 
create the 
> dummy client definitions).
> - remove the dummy definitions
> - recreate the clients with the original names (i.e. use original 
> client_name/clientID values)

I hesitate to do something so drastic just yet, and without direct 
instruction from EMC support.

I just checked, and told the GUI SQL recover to do a NORMAL restore. Then 
I told it to change Browse time. And it showed *just* last night's backup, 
9/27/2011, in the list of backup versions to choose. If I clicked on 
"Browse Time", it said "Time range must be between 9/27/2011 21:35 and 
9/27/2011 21:35". Apparently that's all it thinks is there.

And yet ....

>mminfo -avot -q "client=pssql3,name=MSSQL:,savetime>09/23/2011" -r 
name,totalsize(2),ssid,savetime(22),ssbrowse(22)
 name                       total ssid           date     time browse time
MSSQL:                     265 GB 3833408418  9/23/2011 9:00:18 PM 
11/23/2011 11:59:59 PM
MSSQL:                    6151 MB 3229430570  9/23/2011 9:32:21 PM 
11/23/2011 11:59:59 PM
MSSQL:                    3284 KB 3145544545  9/23/2011 9:33:20 PM 
11/23/2011 11:59:59 PM
MSSQL:                    3093 KB 3128767331  9/23/2011 9:33:23 PM 
11/23/2011 11:59:59 PM
MSSQL:                    1495 KB 3111990118  9/23/2011 9:33:25 PM 
11/23/2011 11:59:59 PM
MSSQL:                      14 MB 3078435688  9/23/2011 9:33:27 PM 
11/23/2011 11:59:59 PM
MSSQL:                     440  B 3061658477  9/23/2011 9:33:31 PM 
11/23/2011 11:59:59 PM
MSSQL:                     265 GB 1249976353  9/26/2011 9:00:17 PM 
11/26/2011 11:59:58 PM
MSSQL:                    6395 MB 176236403   9/26/2011 9:31:22 PM 
11/26/2011 11:59:59 PM
MSSQL:                    3604 KB 159459244   9/26/2011 9:32:27 PM 
11/26/2011 11:59:59 PM
MSSQL:                    3093 KB 142682030   9/26/2011 9:32:29 PM 
11/26/2011 11:59:59 PM
MSSQL:                    1496 KB 125904816   9/26/2011 9:32:31 PM 
11/26/2011 11:59:59 PM
MSSQL:                      15 MB 109127602   9/26/2011 9:32:34 PM 
11/26/2011 11:59:59 PM
MSSQL:                     440  B 92350390    9/26/2011 9:32:37 PM 
11/26/2011 11:59:59 PM
MSSQL:                     265 GB 729969055   9/27/2011 9:00:14 PM 
3/27/2012 11:59:59 PM
MSSQL:                    6674 MB 3699538309  9/27/2011 9:33:54 PM 
3/27/2012 11:59:59 PM
MSSQL:                    3668 KB 3598875071  9/27/2011 9:34:55 PM 
3/27/2012 11:59:59 PM
MSSQL:                    3093 KB 3582097857  9/27/2011 9:34:57 PM 
3/27/2012 11:59:59 PM
MSSQL:                    1496 KB 3565320644  9/27/2011 9:34:59 PM 
3/27/2012 11:59:59 PM
MSSQL:                      15 MB 3548543430  9/27/2011 9:35:02 PM 
3/27/2012 11:59:59 PM
MSSQL:                     440  B 3531766218  9/27/2011 9:35:05 PM 
3/27/2012 11:59:59 PM

Here we see SQL databases being backed up before 9/27/2011, with browse 
time at least 2 months in the future (I changed browse time to 6 months 
yesterday, thinking perhaps that changing it would kick something loose 
and maybe make it start to work). Yet I can't browse to them, to do a 
restore.

If I do a mminfo for the actual index, I see entries that seem the same 
size, for 9/27 and before (and a large one for Friday night that I 
recovered, which would be a non-SQL files backup ...). Every night we do 
SQL backups, and regular filesystem backups, from this client, so I guess 
that's why a normal night's index is 700-800K. But I don't know how to 
check the contents of the index, to see what files those records pertain 
to. Apparently, they don't pertain to the SQL backups listed above, since 
I couldn't browse to a SQL backup before 9/27/2011.

>mminfo -avot -q "name=index:pssql3,savetime>09/22/2011" -r 
name,totalsize(2),ssid,savetime(22),ssbrowse(22)
 name                       total ssid           date     time browse time
index:pssql3               796 KB 2205891239  9/22/2011 9:39:51 AM 
11/22/2011 11:59:59 PM
index:pssql3               851 KB 1232900019  9/23/2011 9:54:59 AM 
11/23/2011 11:59:59 PM
index:pssql3                12 MB 3263030167 9/24/2011 10:05:11 AM 
11/24/2011 11:59:59 PM
index:pssql3               820  B 2206238424 9/26/2011 10:06:16 AM 
11/26/2011 11:59:59 PM
index:pssql3               731 KB 2004997867 9/27/2011 10:00:11 AM 
11/27/2011 11:59:59 PM


There is an index backup going on right now, as a matter of fact - we do 
it daily at 9AM - so I will check back later, and see if the index is 
still there ...

The other client I was having trouble with, the one where the index was 
actually zero, has been offline for a couple days now (hardware failure). 
So I can't do anything more with it ...

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the body of the email. Please write to 
networker-request AT listserv.temple DOT edu if you have any problems with this 
list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER