Networker

Re: [Networker] Savesets being deleted -- how to catch it???

2011-03-06 09:08:21
Subject: Re: [Networker] Savesets being deleted -- how to catch it???
From: jee <jee AT ERESMAS DOT NET>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Date: Sun, 6 Mar 2011 14:08:00 +0000
HI Francis,

Have you checked the daemon.log?  Saveset deletions should be logged on the 
daemon.log.  (sorry, I have to ask , you didn't mention this bit)

I suggest to add ssflags, sumflags and clflags to the mminfo report just in 
case. If the savesets are not good then this may explain why they are being 
removed (automatically).

Regarding remote connections:

(1) NW commands

Can you use IPtables on the Linux server to drop and log connections from the 
Oracle client to the server during the time frame when the savesets are being 
deleted? (I would block the full range of Service ports just in case) 

- If the savesets are still deleted during that time frame, then you can 
discard those connections as the source of the problem (and if they have been 
attempted -- check the Iptables log -- then they may have tried to do 
something unrelated to this problem)

- If the savesets are not deleted and the connections are still being atempted 
(and dropped) , then you can go back again to the Oracle DBAs and let them 
know about that because those connections would be most likely the culprits. 


(2)  Remote shell commands:

If someone is remotely running some scripts on the linux server from the AIX 
client using ssh or rsh/remsh, then a wrapper on the AIX box can be used to 
identify the AIX user, the linux script (or the AIX command -- if inline)  
and the time when the remote command is beig executed.

Note: You may need to move remsh/rsh or ssh to a different location because 
they may not  like to be renamed. In my case, I created the dir /usr/bin/BIN 
and moved remsh to it. Then I wrote the wrapper script "remsh" 
under /usr/bin. The wrapper would print the user id, the remote command and 
the timestamp to to a log file. Then it would exec the actual binary with the 
original parameters using:
 exec /usr/bin/BIN/remsh "$@"



I hope this may  help
jee


On Thursday 03 March 2011 16:56:45 Francis Swasey wrote:
> We have a customer group (Oracle DBA's) who refuse to let the NetWorker
> server run their backups.  They have scripts that perform the Oracle
> backups then run a save command from the scripts to send the backups into
> NetWorker.
>
> For the past two days, I've been trying to help them track down how they
> are getting deleted. The first day, I set up a cron job that used mminfo
> and verified the existence of the saveset every hour.  The saveset that had
> been written at 6am with a browse time of 1 year and a retention of 7 years
> disappeared between 2pm and 3pm.  The second day, I checked every two
> minutes between 2pm and 3pm and the saveset that had been written that
> morning was deleted between 2:34pm and 2:36pm.
>
> Now, I was running a tcpdump during this time catching all traffic to the
> ports that nsrexecd was listening to on the NetWorker server (assuming that
> whatever client is doing this delete would be talking to nsrexecd to do
> it).  I found a single one of their machines that was talking during the
> whole time -- but the DBA's are insistent there was no script that was
> running in that two minute window that "should" have deleted the saveset.
>
> Is there a way that I can get NetWorker to log when a saveset is deleted? 
> I'm assuming that some script somewhere is running an nsrmm command (Thus,
> the NMC logging is not going to help me here).  Is there any other command
> that could be run that would delete a saveset?
>
> The server is running 7.5.3.4 on Red Hat Enterprise Linux 5 (64-bit) -- if
> that makes any difference in the answer.  The client I think is guilty is
> running AIX 5.3 and NetWorker 7.3.3.
>
> Thanks,

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the body of the email. Please write to 
networker-request AT listserv.temple DOT edu if you have any problems with this 
list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

<Prev in Thread] Current Thread [Next in Thread>