Networker

Re: [Networker] Limit on number of savesets in an nsrclone?

2008-03-10 09:49:30
Subject: Re: [Networker] Limit on number of savesets in an nsrclone?
From: Ian G Batten <ian.batten AT UK.FUJITSU DOT COM>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Date: Mon, 10 Mar 2008 13:42:54 +0000
It's not the size of the list: I'm seeing failures at 25 (I've modified my script to pre-split the input into batches of 25)

What's happening is that the nsrclone starts, the adv_file and the tape device are mounted, and then the nsrclone fails. The fact that (in this example) the volume mounts have been notified indicates that things have progressed, and the session is actually appearing in the GUI as in progress (briefly). You can see in this nsrwatch output that the save sessions actually come up, and then immediately `complete'. I'm worried by the flurry of nsrmmd restarts, though.

ian



Mon 13:37:24 media info: restarting nsrmmd #4 on backup-srv.ftel.co.uk in 2 minute(s) Mon 13:37:25 media waiting event: Waiting for 1 writable volumes to backup pool 'Incrementals' disk(s) or tape(s) on sonic.ftel.co.uk Mon 13:37:25 /var/remotestage/incrementals/offsite-4/_AF_readonly mounted adv_file disk IncrementalsStaging.003.RO (write protected) Mon 13:37:50 media info: restarting nsrmmd #3 on backup- srv.ftel.co.uk now Mon 13:37:50 rd=sonic.ftel.co.uk:/dev/rmt/3cbn Verify label operation in progress Mon 13:37:54 rd=sonic.ftel.co.uk:/dev/rmt/3cbn verified label of 002455L3 Mon 13:37:54 rd=sonic.ftel.co.uk:/dev/rmt/3cbn Mount operation in progress Mon 13:38:12 rd=sonic.ftel.co.uk:/dev/rmt/3cbn mounted LTO Ultrium-2 tape 002455L3 Mon 13:38:12 media event cleared: Waiting for 1 writable volumes to backup pool 'Incrementals' disk(s) or tape(s) on sonic.ftel.co.uk Mon 13:38:12 cloning session:1 of 25 save set(s) starting read from IncrementalsStaging.003.RO of 915 MB Mon 13:38:12 write completion notice: Writing to volume 002456L3 complete
 Mon 13:38:13 sonic.ftel.co.uk:cloning session done saving
Mon 13:38:13 cloning session:25 of 25 save set(s) done reading from IncrementalsStaging.003.RO Mon 13:38:13 /var/remotestage/incrementals/offsite-4/_AF_readonly enabled; nsrmmd not available Mon 13:38:13 media info: restarting nsrmmd #11 on backup- srv.ftel.co.uk in 2 minute(s) Mon 13:38:14 /var/remotestage/incrementals/offsite-4/_AF_readonly mounted adv_file disk IncrementalsStaging.003.RO (write protected) Mon 13:38:20 media info: nsrmmd #3 on backup-srv.ftel.co.uk started as requested Mon 13:38:41 cloning session:1 of 25 save set(s) starting read from IncrementalsStaging.003.RO of 915 MB
 Mon 13:38:42 sonic.ftel.co.uk:cloning session done saving
Mon 13:38:42 cloning session:25 of 25 save set(s) done reading from IncrementalsStaging.003.RO Mon 13:38:42 /var/remotestage/incrementals/offsite-4/_AF_readonly enabled; nsrmmd not available Mon 13:38:42 media info: restarting nsrmmd #20 on backup- srv.ftel.co.uk in 2 minute(s) Mon 13:39:26 media info: restarting nsrmmd #4 on backup- srv.ftel.co.uk now Mon 13:39:26 media info: nsrmmd #4 on backup-srv.ftel.co.uk started as requested Mon 13:40:18 media info: restarting nsrmmd #11 on backup- srv.ftel.co.uk now Mon 13:40:18 media info: nsrmmd #11 on backup-srv.ftel.co.uk started as requested Mon 13:40:47 write completion notice: Writing to volume 002357L3 complete Mon 13:40:51 media info: restarting nsrmmd #20 on backup- srv.ftel.co.uk now Mon 13:40:51 media info: nsrmmd #20 on backup-srv.ftel.co.uk started as requested





On 09 Mar 08, at 0733, Vincent Lin wrote:

On one of my 7.2.2 build 494 Networker servers,
I've got 1,700+ savesets devided into 2 tape pools across 4 x 1 TB AFTDs. I break them down to some small batches, 100 savesets per run per LTO2 tape drive over 4 drives.
It is much more manageable this way.

----- Original Message ----
From: Peter Viertel <Peter.Viertel AT MACQUARIE DOT COM>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Sent: Sunday, March 9, 2008 12:26:09 PM
Subject: Re: [Networker] Limit on number of savesets in an nsrclone?

I clone absolutely everything by ssid under some of my 7.2.2.494
systems. I've bigger lists than that in by file and by stdin without any
issues. Just looking at recent tempfiles I have lying around - one has
528 ssid's in it and adds up to 11435 bytes, and that went through just fine. Not to say there's not a bug in 7.3.3 though. I will be doing
a big batch on a 7.3.3 system this afternoon and will keep Ian's
reported problem in mind for that.

-----Original Message-----
From: EMC NetWorker discussion [mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU] On
Behalf Of Werth, Dave
Sent: Saturday, 8 March 2008 4:37 AM
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Subject: Re: [Networker] Limit on number of savesets in an nsrclone?

I don't know if this applies but often there is a limit on the length of a command line. In Solaris I believe it is 2048 bytes. So even though you're putting the savesets to be cloned in a file it may be parsing it
into a command line as if you had tried to write it all without the
savesets file. So it's not so much the number of savesets you can clone but the total length of the command line that results. For instance if
your savesets were name a, b, c, etc. you might be able to clone well
over 1000 savesets.

Dave

Dave Werth
Garmin AT, Inc.
Salem, Oregon
-----Original Message-----
From: EMC NetWorker discussion [mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU] On
Behalf Of Ian G Batten
Sent: Friday, March 07, 2008 8:35 AM
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Subject: [Networker] Limit on number of savesets in an nsrclone?

I'm in the process of binary chopping to find the limit, but it
appears that there is a limit to the number of savesets I can
reference in an nsrclone -S -f /file/name (Networker 7.3.3 on Solaris
10).  175 is OK, while 250 is too many.

I have a script which implements a policy of writing a daily tape of
activity in a given pool, but not deleting the files until a week has
passed.  Recently we wrote a lot of savesets during one day, taking
the amount of activity to go to tape at night over the magic
threshold, from which there is no return.  nsrwatch shows the clone
job start (cloning session: XXX save sets  reading from...) but then
immediately reports completion, while the client sees the messages
below.

/etc/tools/clonetotape.test: saving 313 savesets from
IncrementalsStaging.003.RO to Incrementals...
+ nsrclone -b Incrementals -s backup-srv.ftel.co.uk -S -f /tmp/
clonetotape.test.25004
nsrclone: RPC error: RPC receive operation failed.  A network
connection could not be established with the host.
nsrclone: Cannot open nsrclone session with backup-srv.ftel.co.uk
nsrclone: Cannot open nsrclone session with backup-srv.ftel.co.uk.
Error is 'RPC receive operation failed.  A network connection could
not be established with the host.'
nsrclone: Failed to clone any save sets

To sign off this list, send email to listserv AT listserv.temple DOT edu and
type "signoff networker" in the body of the email. Please write to
networker-request AT listserv.temple DOT edu if you have any problems with this
list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

-------------------------
This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient. If you are not the intended
recipient, please be aware that any disclosure, copying, distribution or use of this e-mail or any attachment is prohibited. If you have received
this e-mail in error, please contact the sender and delete all copies.
Thank you for your cooperation

To sign off this list, send email to listserv AT listserv.temple DOT edu and
type "signoff networker" in the body of the email. Please write to
networker-request AT listserv.temple DOT edu if you have any problems with this
list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

NOTICE
This e-mail and any attachments are confidential and may contain copyright material of Macquarie Group Limited or third parties. If you are not the intended recipient of this email you should not read, print, re-transmit, store or act in reliance on this e-mail or any attachments, and should destroy all copies of them. Macquarie Group Limited does not guarantee the integrity of any emails or any attached files. The views or opinions expressed are the author's own and may not reflect the views or opinions of Macquarie Group Limited.

To sign off this list, send email to listserv AT listserv.temple DOT edu and type "signoff networker" in the body of the email. Please write to networker-request AT listserv.temple DOT edu if you have any problems with this list. You can access the archives at http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER






____________________________________________________________________________________
Be a better friend, newshound, and
know-it-all with Yahoo! Mobile.  Try it now.  
http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

To sign off this list, send email to listserv AT listserv.temple DOT edu and type "signoff networker" in the body of the email. Please write to networker-request AT listserv.temple DOT edu if you have any problems with this list. You can access the archives at http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

To sign off this list, send email to listserv AT listserv.temple DOT edu and type 
"signoff networker" in the body of the email. Please write to networker-request 
AT listserv.temple DOT edu if you have any problems with this list. You can access the 
archives at http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER