ADSM-L

Re: [ADSM-L] How to delete a node/filespaces when target replica server is unavailable

2016-09-01 10:34:53
Subject: Re: [ADSM-L] How to delete a node/filespaces when target replica server is unavailable
From: Zoltan Forray <zforray AT VCU DOT EDU>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Thu, 1 Sep 2016 10:33:09 -0400
I bounced the server and successfully turned off replication for the node
with the problem and deleted it's filespaces.

However, now I can't get ANY replication to run - even when trying to
replicate a single node.  I get these errors (I have tried Googling but did
not find much and what I did find talked about earlier versions of TSM
server):

9/1/2016 10:30:14 AM ANR2017I Administrator ZFORRAY issued command:
REPLICATE NODE orarcat.vcu.edu maxsessions=2
9/1/2016 10:30:15 AM ANR2032E REPLICATE NODE: Command failed - internal
server error detected.
9/1/2016 10:30:15 AM ANR9999D Thread<846> issued message 2032 from:
9/1/2016 10:30:15 AM ANR9999D Thread<846>  0x00000000dde43a outTextf
9/1/2016 10:30:15 AM ANR9999D Thread<846>  0x0000000087cb66
BuildFSReplicationList
9/1/2016 10:30:15 AM ANR9999D Thread<846>  0x0000000087e3ee imReplicateNode
9/1/2016 10:30:15 AM ANR9999D Thread<846>  0x000000005a0506
AdmReplicateNode
9/1/2016 10:30:15 AM ANR9999D Thread<846>  0x000000004eb03e AdmCommandLocal
9/1/2016 10:30:15 AM ANR9999D Thread<846>  0x000000004ec28e admCommand
9/1/2016 10:30:15 AM ANR9999D Thread<846>  0x00000000ca7fca
SmAdminCommandThread
9/1/2016 10:30:15 AM ANR9999D Thread<846>  0x00000000e7c6ea StartThread
9/1/2016 10:30:15 AM ANR9999D Thread<846>  0x00003b7d207aa1 *UNKNOWN*
9/1/2016 10:30:15 AM ANR9999D Thread<846>  0x00003b7cee8aad *UNKNOWN*
9/1/2016 10:30:15 AM ANR1930E REPLICATE NODE: There are no eligible
filespaces for node ORARCAT.VCU.EDU.
9/1/2016 10:30:15 AM ANR9999D_0413043177 GetCommandRc(admrepl.c:4727)
Thread<846>: unexpected rc=1114
9/1/2016 10:30:15 AM ANR9999D Thread<846> issued message 9999 from:
9/1/2016 10:30:15 AM ANR9999D Thread<846>  0x00000000de4373 OutDiagToCons
9/1/2016 10:30:15 AM ANR9999D Thread<846>  0x00000000de7175 outDiagfExt
9/1/2016 10:30:15 AM ANR9999D Thread<846>  0x0000000059c6e4 GetCommandRc
9/1/2016 10:30:15 AM ANR9999D Thread<846>  0x000000005a025f
AdmReplicateNode
9/1/2016 10:30:15 AM ANR9999D Thread<846>  0x000000004eb03e AdmCommandLocal
9/1/2016 10:30:15 AM ANR9999D Thread<846>  0x000000004ec28e admCommand
9/1/2016 10:30:15 AM ANR9999D Thread<846>  0x00000000ca7fca
SmAdminCommandThread
9/1/2016 10:30:15 AM ANR9999D Thread<846>  0x00000000e7c6ea StartThread
9/1/2016 10:30:15 AM ANR9999D Thread<846>  0x00003b7d207aa1 *UNKNOWN*
9/1/2016 10:30:15 AM ANR9999D Thread<846>  0x00003b7cee8aad *UNKNOWN*


On Thu, Sep 1, 2016 at 8:58 AM, Zoltan Forray <zforray AT vcu DOT edu> wrote:

> I forgot to mention I already did that but while it was stuck issuing
> errors. Just tried it again and it confirms it is not being replicated.
> Time to restart the TSM server. Hoping I won't have to bounce the whole
> box. The last time a replication process got stuck on a target server,
> halting the TSM server would not clear it.  The server would not shut down
> so I bounced the whole thing.
>
> 8:51:18 AM   HADES : remove replnode compile.vcu.edu
> ANR1630W REMOVE REPLNODE: COMPILE.VCU.EDU is not configured for
> replication.
> ANR2679E REMOVE REPLNODE: All of the nodes provided are invalid.
> ANR1632E REMOVE REPLNODE: Command failed. Replication state information
> for
> the specified nodes could not be removed.
> ANS8001I Return code 11.
>
>
> On Thu, Sep 1, 2016 at 8:40 AM, David Ehresman <
> david.ehresman AT louisville DOT edu> wrote:
>
>> On the source server, try "remove replnode NODENAME", then try deleting
>> its filespaces.
>>
>> David
>>
>> -----Original Message-----
>> From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf 
>> Of
>> Zoltan Forray
>> Sent: Thursday, September 01, 2016 8:26 AM
>> To: ADSM-L AT VM.MARIST DOT EDU
>> Subject: [ADSM-L] How to delete a node/filespaces when target replica
>> server is unavailable
>>
>> We are just starting to roll out replication and I have a problem.
>>
>> Many months ago, I tested replication/configuration on a single node.  Now
>> I need to delete the node and all of its filespaces.
>>
>> But the target server used for replication testing has been taken
>> down/unavailable.
>>
>> Now when I issued delete filespaces for the node on the source server, the
>> delete process has hung. Attempting to cancel the delete process has been
>> hung for >24-hours and is starting to cause log pinning issues.
>>
>> I am ready to bounce this server (RH Linux with 6.3.5.100 server) but
>> still
>> don't know how to delete these filespaces without hanging things up,
>> again.
>>
>> I have turned off replication on the source server. Tried to update the
>> node to disable replication but that didn't seem to help/take.
>>
>> --
>> *Zoltan Forray*
>> TSM Software & Hardware Administrator
>> Xymon Monitor Administrator
>> VMware Administrator (in training)
>> Virginia Commonwealth University
>> UCC/Office of Technology Services
>> www.ucc.vcu.edu
>> zforray AT vcu DOT edu - 804-828-4807
>> Don't be a phishing victim - VCU and other reputable organizations will
>> never use email to request that you reply with your password, social
>> security number or confidential personal information. For more details
>> visit https://urldefense.proofpoint.com/v2/url?u=http-3A__infosecu
>> rity.vcu.edu_phishing.html&d=AwIBaQ&c=SgMrq23dbjbGX6e0ZsSHg
>> EZX6A4IAf1SO3AJ2bNrHlk&r=dOGCMY197NTNH1k_wcsrWS3_fxedKW4rpKJ
>> 8cHCD2L8&m=q25GyWPNnS9scC8LnF8VhVZP_CBMMjSimusH4Kjy5gk&s=T7Y
>> D4SfFzcqmGHMpT9uZYJqBhEZ3v5MaKBWoelfQ5Pc&e=
>>
>
>
>
> --
> *Zoltan Forray*
> TSM Software & Hardware Administrator
> Xymon Monitor Administrator
> VMware Administrator (in training)
> Virginia Commonwealth University
> UCC/Office of Technology Services
> www.ucc.vcu.edu
> zforray AT vcu DOT edu - 804-828-4807
> Don't be a phishing victim - VCU and other reputable organizations will
> never use email to request that you reply with your password, social
> security number or confidential personal information. For more details
> visit http://infosecurity.vcu.edu/phishing.html
>



--
*Zoltan Forray*
TSM Software & Hardware Administrator
Xymon Monitor Administrator
VMware Administrator (in training)
Virginia Commonwealth University
UCC/Office of Technology Services
www.ucc.vcu.edu
zforray AT vcu DOT edu - 804-828-4807
Don't be a phishing victim - VCU and other reputable organizations will
never use email to request that you reply with your password, social
security number or confidential personal information. For more details
visit http://infosecurity.vcu.edu/phishing.html

<Prev in Thread] Current Thread [Next in Thread>