Networker

Re: [Networker] Indexes disappearing ..

2011-09-27 14:51:59
Subject: Re: [Networker] Indexes disappearing ..
From: George Sinclair <George.Sinclair AT NOAA DOT GOV>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Date: Tue, 27 Sep 2011 14:51:34 -0400
On 2011-09-27 13:55, Michael Leone wrote:
Here's a weird one .. I have 2 clients (both virtual cluster clients),
both with browse and retention policies of 2 months, who have their
indexes either completely deleted (i.e., the index sub-folder for that
client is 0K),

I'm confused about this part: "is 0K". How is it OK if it's had its indexes completely deleted? Are you saying that the directory itself if still there but not its contents?

 or it only has the previous day's activity. These are both
backups made with the SQL Module. Neither client nor job has changed in at
least months, perhaps longer.

Even weirder - this seems to happen on every weekday. The index backup (we
do "savegrp -O") on Sat morning - which would have the index from the
Friday backup - would be there. And that would be the only index backup
that wasn't empty.

mminfo -avot -q
"client=networker-server,name=index:client,savetime>09/01/2011" -r
volume,client,ssid,name,level,savetime(22),totalsize(2)

  volume        client     ssid       name                             lvl
   date     time total
006792         networker-server 1667207181 index:client full 9/1/2011
10:00:45 AM 32 KB
006872         networker-server 1298192353 index:client full 9/2/2011
9:19:29 AM 32 KB
006872         networker-server 2825005238 index:client full 9/3/2011
9:16:38 AM 10 MB

I wouldn't have noticed it, except that I needed to do a recover, and I
couldn't because it said there had been no files backed up (which I knew
to be false).
(that snippet is for earlier in the month, but it's the same all month)

"nsrls" shows the total size of the index to be zero
"nsrck -L6" shows no errors, and a total size of zero
"nsrck -L7" reads back in zero (unless I specify one of the Saturday
backups, like 9/3/2011 above)


Is it possible that the database client has been given permission to delete client index entries? I'm not sure how things work with the SQL module, but for the Oracle module, that's exactly how it works. With Oracle, the recommendation is to give the client administrative permissions to be able to remove its CFI entries. Apparently, it will remove only its own entries (we hope) and not some other client's entries. The theory is that you want the catalog (for example) on the Oracle client to be in sync with the CFI entries on the backup server side. If you don't do this, you will see error messages in the NetWorker log file like:

Permission denied, user 'oracle' on 'clientxyz' does not have 'Operate NetWorker' privilege to perform this operation - NSR.

And you will see error message in the Oracle logs on the client like:

lnm_index_remove_SSID: Permission denied, user 'oracle' on 'clientxyz' does not have 'Operate NetWorker' privilege to perform this operation - NSR.

Perhaps, you have the time window set to 1 day on the SQL client for the catalog, and it's going out and removing those index entries? Something like that?

Note that the groups that these 2 clients are in are set to "no index
save", like all our groups. I have 130 clients, and am only seeing this
problem on these 2. One client is a clone job, one is not.
EMC seems to be confused, as well. :-)

I don't see why that would be an issue. That controls whether or not the client index gets written to tape. If you have it enabled for the pool, that allow the entries to get added to the CFI on disk, right?


Thoughts, anyone? I am going to uncheck the "No index save" temporarily on
these groups. Can anyone suggest something else to diagnose?

Thanks



--
George Sinclair
Voice: (301) 713-3284 x210
- The preceding message is personal and does not reflect any official or unofficial position of the United States Department of Commerce -
- Any opinions expressed in this message are NOT those of the US Govt. -

To sign off this list, send email to listserv AT listserv.temple DOT edu and type 
"signoff networker" in the body of the email. Please write to networker-request 
AT listserv.temple DOT edu if you have any problems with this list. You can access the 
archives at http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER