Veritas-bu

[Veritas-bu] Multiple data stream and bpend_notify script

2005-11-08 21:26:49
Subject: [Veritas-bu] Multiple data stream and bpend_notify script
From: pcd AT xinupro DOT com (Peter DrakeUnderkoffler)
Date: Tue, 08 Nov 2005 21:26:49 -0500
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

The times I've written pre/post scripts taking multiple data streams into
account, I used mkdir to prevent race conditions as it is atomic
where touch or other file making/updating programs are not.


Thanks
Peter DrakeUnderkoffler

Mark.Donaldson AT cexp DOT com wrote:
> There's the very slight chance of the race condition but it's really slim.
> 
> Since it appends only a single line to a file, the two streams would
> literally have both end at near the same time.
> 
> The normal order would be:
> 
> write
> check (fail)
> write
> check (succeed)
> 
> The race condition would have to be:
> 
> write
> write
> check (succeed)
> check (succeed)
> 
> And that's really unlikely.
> 
> If you're worried, and since I wrote the stream number to the count-file,
> then you could check which stream wrote to the file last:
> 
> echo $STREAM_NUMBER >>$COUNTILFE
> if [ `wc -l $COUNTFILE|awk '{print $1}' -eq $STREAM_COUNT -a \
>        `tail -1 $COUNTFILE` -eq $STREAM_NUBMER ]
> then
>   #count is correct and this stream wrote the last line
>   rm $COUNTFILE
>   ## Last stream processing
> fi
> 
> Even more paranoia, you could use a monolithic command like mkdir to create
> a lockfile that would only allow one stream to exit at the same time.
> 
> HTH - M
> 
> 
> -----Original Message-----
> From: veritas-bu-admin AT mailman.eng.auburn DOT edu
> [mailto:veritas-bu-admin AT mailman.eng.auburn DOT edu]On Behalf Of Guang Yu 
> Liu
> Sent: Tuesday, November 08, 2005 5:03 PM
> To: 'Mark.Donaldson AT cexp DOT com'; Asiye.Yigit AT gantek DOT com;
> veritas-bu AT mailman.eng.auburn DOT edu
> Subject: RE: [Veritas-bu] Multiple data stream and bpend_notify script
> 
> 
> wouldn't there be a chance that there might be race condition issues?
> Suppose that two processes can modify the file at the same time and that
> could corrupt the countfile.
> Before testing NBU6 in the future we will just use the old flat file backup
> (cron jobs dump the db to flat files). 
> -----Original Message----- 
> From: Mark.Donaldson AT cexp DOT com [mailto:Mark.Donaldson AT cexp DOT com] 
> Sent: Wednesday, November 09, 2005 3:19 AM 
> To: Asiye.Yigit AT gantek DOT com; veritas-bu AT mailman.eng.auburn DOT edu 
> Subject: RE: [Veritas-bu] Multiple data stream and bpend_notify script 
> 
> 
> In unix, I do this with what I call a "count" file. 
> As each stream ends, it writes appends a line to a common file.  The script 
> then checks the number of lines in the file and compares it against the 
> "STREAM_COUNT" variable.  If they match then the end-job is run & the count 
> file removed. 
> Beats me how to do this in MS Batch files but here's my script's logic: 
> echo $STREAM_NUMBER >> /tmp/countfile 
> if [ `wc -l /tmp/countfile | awk '{print $1}'` -eq $STREAM_COUNT ] 
> then 
>   echo "Last stream" 
>   rm /tmp/countfile 
>   <run end-of-job-stuff> 
> fi 
> 
> 
> There's some additional logic in the bpstart_notify script to check for an 
> old countfile that's gotten "lost" because a previous stream ended without 
> running it bpend_notify script.  This, of course, may lead to the end-of-job
> 
> startup stuff not running.  Usually, for me, that's not an issue. 
> -M 
> -----Original Message----- 
> From: veritas-bu-admin AT mailman.eng.auburn DOT edu 
> [mailto:veritas-bu-admin AT mailman.eng.auburn DOT edu]On Behalf Of Asiye 
> Yigit 
> Sent: Tuesday, November 08, 2005 2:37 AM 
> To: veritas-bu AT mailman.eng.auburn DOT edu 
> Subject: [Veritas-bu] Multiple data stream and bpend_notify script 
> 
> 
> Hi gurus, 
> We have a file system backup. It is being backuped as multiple data 
> stream. 
> We are backing up offline sql backup. So, in the bpend_notify script, 
> There is start command to start sql after finishing backup. 
> As there are multiple data stream, after completion of each data stream, 
> Nb run bpend_notify script. We have to instruct the script, 
> There are multiple data stream. In fact, in the admin guide, 
> We have found some variables to support multiple data stream: 
> STREAM_NUMBER 
> STREAM_COUNT 
> However, I do not know how can I use these variables in my bpend_notify 
> script. 
> Does  "Set STREAM_NUMBER=1"  work? I am not sure. How can I tell the 
> script, 
> This backup is being backed up via multiple data stream. Please wait? 
> Is there any idea about that? I would appreciate if you could help me. 
> Regards, 
> Asiye 
> 
> 
> _______________________________________________ 
> Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu 
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu 
> Macquarie Telecom - delivering value for business and government. 
> ----------------------------------------------------------------------------
> ----------------------------------- 
> Confidentiality Note: This e-mail is sent to and intended for use by the
> named addressees only. 
> It contains confidential information. If you receive this e-mail in error,
> please telephone Macquarie 
> Telecom Pty Ltd on +612 8221 7777, and then delete this message immediately.
> Further, you should 
> not re-transmit, copy, store, or reveal the contents of this message to any
> third party. 
> _______________________________________________
> Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (FreeBSD)

iD8DBQFDcV5pl+lekZRM55oRAlWNAJ9HGpcLgHeygHqwZ39EOvR0t5BwqgCgswss
2YXHHAEqpEYQjp1K4AREGIQ=
=AqC0
-----END PGP SIGNATURE-----