Veritas-bu

Re: [Veritas-bu] Strange RMAN Behavior

2010-04-29 16:55:58
Subject: Re: [Veritas-bu] Strange RMAN Behavior
From: "Stafford, Geoff" <GStafford AT barclaycardus DOT com>
To: "Mark Glazerman" <Mark.Glazerman AT spartech DOT com>, veritas-bu AT mailman.eng.auburn DOT edu
Date: Thu, 29 Apr 2010 16:54:46 -0400
Are you starting via cron or elsewhere?  I don't see where you are passing a 
send statement.  If you don't define the parameter than it will run the backup 
in the first policy of type Oracle it finds (in alphabetical order) with the 
client in that policy.  Everything is running in the same policy.

Note this from your original email for an archive:

4/29/2010 2:53:23 PM - requesting resource
nbumaster01.sun.spartech.com.NBU_POLICY.MAXJOBS.Solaris_OraArchTier1

And this from the level 0:

4/17/2010 12:59:59 AM - requesting resource
nbumaster01.sun.spartech.com.NBU_POLICY.MAXJOBS.Solaris_OraDbTier1

Both show the same policy.

And this from the most recent email showing the full running in that policy:

NB_ORA_POLICY: Solaris_OraDbTier1

Here's an example of a send statement:

SEND 
'NB_ORA_CLIENT=client_name,NB_ORA_POLICY=policy_name,NB_ORA_SERV=master_name,NB_ORA_SCHED=sched_name';

If you aren't defining these variables you are pretty much just praying that it 
will go the right place.  Since you have one script for both types of backups 
you'll have to do appropriate if statements to figure out which policy you want.

Also, I'm not a fan of defining filesperset in archive log jobs, it makes the 
backup sets smaller and creates more jobs in NBU which means more resource 
broker work, etc etc but do your research on removing it before listening to 
some random guy on the internet.  ;)  Basically your archive log jobs will be 
about 3x bigger assuming there are enough files out there to make the job that 
big.

-----Original Message-----
From: Mark Glazerman [mailto:Mark.Glazerman AT spartech DOT com] 
Sent: Thursday, April 29, 2010 4:36 PM
To: Stafford, Geoff; veritas-bu AT mailman.eng.auburn DOT edu
Subject: RE: Strange RMAN Behavior

Here is our RMAN backup script

[root@oraprod] / > cat /usr/openv/netbackup/bin/oracle_backup.sh
#!/bin/sh
# $Header: oracle_backup.sh $
#
#
# ---------------------------------------------------------------------------
#               Insert Changes made here, along with the date:
# ---------------------------------------------------------------------------
#
#       Copied from hot_database_backup.sh provided by Veritas: 03/08/05
#
# ---------------------------------------------------------------------------
# Determine the user which is executing this script.
# ---------------------------------------------------------------------------

CUSER=`id |cut -d"(" -f2 | cut -d ")" -f1`
ORATAB=/var/opt/oracle/rman_oratab
export ORATAB
x=0

for i in `cat $ORATAB | grep -v \* | grep -v \# | awk -F: '{print $1}'`
do
        ORACLE_SID=""
        export ORACLE_SID
        ORACLE_HOME=""
        export ORACLE_HOME
        TNS_ADMIN=""
        export TNS_ADMIN
        ORACLE_SID=$i
        export ORACLE_SID
        ORACLE_HOME=`cat $ORATAB | grep -v \* | grep -v \# | grep $ORACLE_SID | 
head -1 | awk -F: '{print $2}'`
        export ORACLE_HOME
        if [ -d $ORACLE_HOME/network/admin/$ORACLE_SID'_'`hostname` ];
                then
                TNS_ADMIN=$ORACLE_HOME/network/admin/$ORACLE_SID'_'`hostname`
                else
                TNS_ADMIN=$ORACLE_HOME/network/admin
        fi
        export TNS_ADMIN
        ORACLE_USER=oracle
        export ORACLE_USER

# ---------------------------------------------------------------------------
# Create the Logfile
# Note: output directory requires write permission.
# ---------------------------------------------------------------------------

LOG_DIR=/var/adm/log/bkup_logs
RMAN_LOG_FILE=$LOG_DIR/$ORACLE_SID'_'`date '+%m%d%y'`.out
echo $RMAN_LOG_FILE

# -----------------------------------------------------------------
# Initialize the log file.
# -----------------------------------------------------------------

echo >> $RMAN_LOG_FILE
chmod 666 $RMAN_LOG_FILE

# ---------------------------------------------------------------------------
# Log the start of this script.
# ---------------------------------------------------------------------------

echo "########################################################################" 
>> $RMAN_LOG_FILE
echo "########################################################################" 
>> $RMAN_LOG_FILE
echo "####                                                                ####" 
>> $RMAN_LOG_FILE
echo "####                        BACKUP STARTED                          ####" 
>> $RMAN_LOG_FILE
echo "####                                                                ####" 
>> $RMAN_LOG_FILE
echo "########################################################################" 
>> $RMAN_LOG_FILE
echo "########################################################################" 
>> $RMAN_LOG_FILE

echo $0 initiated backup of $ORACLE_SID >> $RMAN_LOG_FILE
echo ==== started on `date` ==== >> $RMAN_LOG_FILE
echo >> $RMAN_LOG_FILE

# ---------------------------------------------------------------------------
# Set the target connect string.
# Replace "sys/manager", below, with the target connect string.
# ---------------------------------------------------------------------------

TARGET_CONNECT_STR=/

# ---------------------------------------------------------------------------
# Set the Oracle Recovery Manager name.
# ---------------------------------------------------------------------------

RMAN=$ORACLE_HOME/bin/rman

# ---------------------------------------------------------------------------
# Print out the value of the variables set by this script.
# ---------------------------------------------------------------------------

echo >> $RMAN_LOG_FILE
echo   "RMAN: $RMAN" >> $RMAN_LOG_FILE
echo   "ORACLE_SID: $ORACLE_SID" >> $RMAN_LOG_FILE
echo   "ORACLE_USER: $ORACLE_USER" >> $RMAN_LOG_FILE
echo   "ORACLE_HOME: $ORACLE_HOME" >> $RMAN_LOG_FILE
echo   "TNS_ADMIN: $TNS_ADMIN" >> $RMAN_LOG_FILE

# ---------------------------------------------------------------------------
# Print out the value of the variables set by bphdb.
# ---------------------------------------------------------------------------

echo  >> $RMAN_LOG_FILE
echo   "NB_ORA_FULL: $NB_ORA_FULL" >> $RMAN_LOG_FILE
echo   "NB_ORA_INCR: $NB_ORA_INCR" >> $RMAN_LOG_FILE
echo   "NB_ORA_CINC: $NB_ORA_CINC" >> $RMAN_LOG_FILE
echo   "NB_ORA_SERV: $NB_ORA_SERV" >> $RMAN_LOG_FILE
echo   "NB_ORA_POLICY: $NB_ORA_POLICY" >> $RMAN_LOG_FILE

# ---------------------------------------------------------------------------
# NOTE: This script assumes that the database is properly opened. If desired,
# this would be the place to verify that.
# ---------------------------------------------------------------------------

echo >> $RMAN_LOG_FILE
# ---------------------------------------------------------------------------
# If this script is executed from a NetBackup schedule, NetBackup
# sets an NB_ORA environment variable based on the schedule type.
# The NB_ORA variable is then used to dynamically set BACKUP_TYPE
# For example, when:
#     schedule type is                BACKUP_TYPE is
#     ----------------                --------------
# Automatic Full                     INCREMENTAL LEVEL=0
# Automatic Differential Incremental INCREMENTAL LEVEL=1
# Automatic Cumulative Incremental   INCREMENTAL LEVEL=1 CUMULATIVE
#
# For user initiated backups, BACKUP_TYPE defaults to incremental
# level 0 (full).  To change the default for a user initiated
# backup to incremental or incremental cumulative, uncomment
# one of the following two lines.
# BACKUP_TYPE="INCREMENTAL LEVEL=1"
# BACKUP_TYPE="INCREMENTAL LEVEL=1 CUMULATIVE"
#
# Note that we use incremental level 0 to specify full backups.
# That is because, although they are identical in content, only
# the incremental level 0 backup can have incremental backups of
# level > 0 applied to it.
# ---------------------------------------------------------------------------

if [ `ps -ef | grep $ORACLE_SID | grep -v grep | wc -l` -lt 2 ]
        then
                /usr/bin/rm $RMAN_LOG_FILE
        else
                if [ "$NB_ORA_INCR" = "1" -o "$NB_ORA_CINC" = "1" ]
                        then
                                echo "Archive backup requested" >> 
$RMAN_LOG_FILE
                                #Archive log backup
                                CMD_STR="
                                ORACLE_HOME=$ORACLE_HOME
                                export ORACLE_HOME
                                ORACLE_SID=$ORACLE_SID
                                export ORACLE_SID
                                TNS_ADMIN=$TNS_ADMIN
                                export TNS_ADMIN
                                $RMAN target $TARGET_CONNECT_STR catalog 
rman/rman@oem msglog $RMAN_LOG_FILE append << EOF
                                RUN {
                                # backup all archive logs
                                BACKUP
                                #   filesperset 20
                                FORMAT 'b_%u_%p_%c'
                                ARCHIVELOG ALL;
                                }
                                EXIT
                                "
                        else
                                echo "Hot backup requested" >> $RMAN_LOG_FILE
                                #Hotbackup
                                CMD_STR="
                                ORACLE_HOME=$ORACLE_HOME
                                export ORACLE_HOME
                                ORACLE_SID=$ORACLE_SID
                                export ORACLE_SID
                                TNS_ADMIN=$TNS_ADMIN
                                export TNS_ADMIN
                                $RMAN target $TARGET_CONNECT_STR catalog 
rman/rman@oem msglog $RMAN_LOG_FILE append << EOF
                                RUN {
                                BACKUP
                                INCREMENTAL LEVEL=0
                                SKIP INACCESSIBLE
                                #    FILESPERSET 5
                                # recommended format
                                FORMAT 'bk_%s_%p_%t'
                                DATABASE;
                                # backup all archive logs
                                sql 'alter system archive log current';
                                BACKUP
                                filesperset 20
                                FORMAT 'al_%s_%p_%t'
                                ARCHIVELOG ALL DELETE INPUT;
                                }
                                DELETE NOPROMPT OBSOLETE;
                                EXIT
                                "
                fi


                # Initiate the command string
                if [ "$CUSER" = "root" ]
                        then
                                su - $ORACLE_USER -c "$CMD_STR" >> 
$RMAN_LOG_FILE
                                RSTAT=$?
                        else
                                /usr/bin/sh -c "$CMD_STR" >> $RMAN_LOG_FILE
                                RSTAT=$?
                fi

                # 
---------------------------------------------------------------------------
                # Log the completion of this script.
                # 
---------------------------------------------------------------------------

                if [ "$RSTAT" = "0" ]
                        then
                                LOGMSG="ended successfully"
                        else
                                LOGMSG="ended in error"
                fi

                echo >> $RMAN_LOG_FILE
                echo Script $0 >> $RMAN_LOG_FILE
                echo ==== $LOGMSG on `date` ==== >> $RMAN_LOG_FILE
                echo >> $RMAN_LOG_FILE

echo "########################################################################" 
>> $RMAN_LOG_FILE
echo "########################################################################" 
>> $RMAN_LOG_FILE
echo "####                                                                ####" 
>> $RMAN_LOG_FILE
echo "####                        BACKUP ENDED                            ####" 
>> $RMAN_LOG_FILE
echo "####                                                                ####" 
>> $RMAN_LOG_FILE
echo "########################################################################" 
>> $RMAN_LOG_FILE
echo "########################################################################" 
>> $RMAN_LOG_FILE

fi


x=x+1

if [ x = 10 ];
        then
                EXIT 0
fi

done
[root@oraprod] / >

Also... here is the log from the hot database backup that is running every 
morning at 1am but not sending any data anywhere.

[root@oraprod] /var/adm/log/bkup_logs > vi prod1159_042910.out
"prod1159_042910.out" 31880 lines, 1726279 characters

########################################################################
########################################################################
####                                                                ####
####                        BACKUP STARTED                          ####
####                                                                ####
########################################################################
########################################################################
/usr/openv/netbackup/bin/oracle_backup.sh initiated backup of prod1159
==== started on Thu Apr 29 01:00:09 CDT 2010 ====


RMAN: /1159/prod1159db/9.2.0/bin/rman
ORACLE_SID: prod1159
ORACLE_USER: oracle
ORACLE_HOME: /1159/prod1159db/9.2.0
TNS_ADMIN: /1159/prod1159db/9.2.0/network/admin/prod1159_oraprod

NB_ORA_FULL: 1
NB_ORA_INCR: 0
NB_ORA_CINC: 0
NB_ORA_SERV: nbumaster01.sun.spartech.com
NB_ORA_POLICY: Solaris_OraDbTier1

Hot backup requested
Sun Microsystems Inc.   SunOS 5.10      Generic January 2005
You have mail.
RMAN> 2> 3> 4> 5> 6> 7> 8> 9> 10> 11> 12> 13> 14> 15> RMAN> RMAN>  (c) 1995, 
2002, Oracle Corporation.  All rights reserved.

connected to target database: PROD1159 (DBID=3702129868)
connected to recovery catalog database

RMAN> 2> 3> 4> 5> 6> 7> 8> 9> 10> 11> 12> 13> 14> 15>
Starting backup at 29-APR-10
starting full resync of recovery catalog
full resync complete
allocated channel: ORA_SBT_TAPE_1
channel ORA_SBT_TAPE_1: sid=691 devtype=SBT_TAPE
channel ORA_SBT_TAPE_1: Veritas NetBackup for Oracle - Release 6.5 (2007111606)
allocated channel: ORA_SBT_TAPE_2
channel ORA_SBT_TAPE_2: sid=210 devtype=SBT_TAPE
channel ORA_SBT_TAPE_2: Veritas NetBackup for Oracle - Release 6.5 (2007111606)
allocated channel: ORA_SBT_TAPE_3
channel ORA_SBT_TAPE_3: sid=250 devtype=SBT_TAPE
channel ORA_SBT_TAPE_3: Veritas NetBackup for Oracle - Release 6.5 (2007111606)
allocated channel: ORA_SBT_TAPE_4
channel ORA_SBT_TAPE_4: sid=400 devtype=SBT_TAPE
channel ORA_SBT_TAPE_4: Veritas NetBackup for Oracle - Release 6.5 (2007111606)
channel ORA_SBT_TAPE_1: starting incremental level 0 datafile backupset
channel ORA_SBT_TAPE_1: specifying datafile(s) in backupset
input datafile fno=00429 name=/1159/prod1159data/applsysd08.dbf
input datafile fno=00400 name=/1159/prod1159data/applsysd03.dbf
/ENDED
channel ORA_SBT_TAPE_4: sid=312 devtype=SBT_TAPE
channel ORA_SBT_TAPE_4: Veritas NetBackup for Oracle - Release 6.5 (2007111606)
channel ORA_SBT_TAPE_1: starting archive log backupset
channel ORA_SBT_TAPE_1: specifying archive log(s) in backup set
input archive log thread=1 sequence=9073 recid=9067 stamp=717570006
channel ORA_SBT_TAPE_1: starting piece 1 at 29-APR-10
channel ORA_SBT_TAPE_1: finished piece 1 at 29-APR-10
piece handle=b_p2lcafva_1_1 comment=API Version 2.0,MMS Version 5.0.0.0
channel ORA_SBT_TAPE_1: backup set complete, elapsed time: 00:00:56
Finished backup at 29-APR-10

Starting Control File Autobackup at 29-APR-10
piece handle=c-3702129868-20100429-02 comment=API Version 2.0,MMS Version 
5.0.0.0
Finished Control File Autobackup at 29-APR-10

RMAN>

Recovery Manager complete.

Script /usr/openv/netbackup/bin/oracle_backup.sh
==== ended successfully on Thu Apr 29 05:02:15 CDT 2010 ====

########################################################################
########################################################################
####                                                                ####
####                        BACKUP ENDED                            ####
####                                                                ####
########################################################################
########################################################################

Mark Glazerman
Desk: 314-889-8282
Cell: 618-520-3401
 please don't print this e-mail unless you really need to

-----Original Message-----
From: Stafford, Geoff [mailto:GStafford AT barclaycardus DOT com] 
Sent: Thursday, April 29, 2010 3:24 PM
To: veritas-bu AT mailman.eng.auburn DOT edu
Cc: Mark Glazerman
Subject: RE: Strange RMAN Behavior

Can you post the send statement from the rman scripts?  Are you defining
NB_ORA_POLICY?  


Date: Thu, 29 Apr 2010 15:14:25 -0500
From: "Mark Glazerman" <Mark.Glazerman AT spartech DOT com>
Subject: [Veritas-bu] Strange RMAN Behavior
To: <veritas-bu AT mailman.eng.auburn DOT edu>
Message-ID:
        
<6E4842C2D9EBCC4EA5436CC527D38D3607427E67 AT STLCORPEX4.corp DOT dir.spartech.c
om>
        
Content-Type: text/plain; charset="us-ascii"

We have a host that houses an oracle instance backed up by 2 separate
policies.  1 Policy backups up archivelogs.  The second policy backs up
the database.  Both of these policies call the same RMAN script.  Both
policies point to the same storage unit group which contains 2 storage
units.  These storage units are 2 directories on a data domain storage
appliance

 

...snip... 

 

Any help greatly appreciated, 

 

Mark Glazerman



Barclays             www.barclaycardus.com

This e-mail and any files transmitted with it may contain confidential and/or 
proprietary information. It is intended solely for the use of the individual or 
entity who is the intended recipient. Unauthorized use of this information is 
prohibited. If you have received this in error, please contact the sender by 
replying to this message and delete this material from any system it may be on.


Barclays             www.barclaycardus.com

This e-mail and any files transmitted with it may contain confidential and/or 
proprietary information. It is intended solely for the use of the individual or 
entity who is the intended recipient. Unauthorized use of this information is 
prohibited. If you have received this in error, please contact the sender by 
replying to this message and delete this material from any system it may be on.
_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
<Prev in Thread] Current Thread [Next in Thread>