THread Error While starting the instance

[tarunfeb27]

Hello All,

I'm getting below error while starting my TSM instance.
Its a library client hosted on AIX 6.1 server , TSM version is 5.5

121210-202027: ANR9999D Thread<42> issued message 9999 from:
121210-202027: ANR9999D Thread<42> 0x000000010000c7e8 StdPutText
121210-202027: ANR9999D Thread<42> 0x000000010000fb90 OutDiagToCons
121210-202027: ANR9999D Thread<42> 0x000000010000a2d0 outDiagfExt
121210-202027: ANR9999D Thread<42> 0x0000000100124138 StartUtilityModule
121210-202027: ANR9999D Thread<42> 0x0000000100124c60 psOpenFCAdpt
121210-202027: ANR9999D Thread<42> 0x0000000100282320 naDiscoverDevices
121210-202027: ANR9999D Thread<42> 0x0000000100281b1c NaDiscoverThread
121210-202027: ANR9999D Thread<42> 0x00000001000151b0 StartThread
121210-202027: ANR9999D_3612203376 psOpenFCAdpt(psdiscvr.c:426) Thread<42>: Unable to start
121210-202027: SAN discovery utility module
121210-202027: ANR9999D Thread<42> issued message 9999 from:
121210-202027: ANR9999D Thread<42> 0x000000010000c7e8 StdPutText
121210-202027: ANR9999D Thread<42> 0x000000010000fb90 OutDiagToCons
121210-202027: ANR9999D Thread<42> 0x000000010000a2d0 outDiagfExt
121210-202027: ANR9999D Thread<42> 0x0000000100124c88 psOpenFCAdpt
121210-202027: ANR9999D Thread<42> 0x0000000100282320 naDiscoverDevices
121210-202027: ANR9999D Thread<42> 0x0000000100281b1c NaDiscoverThread
121210-202027: ANR9999D Thread<42> 0x00000001000151b0 StartThread
121210-202027: ANR1787W Not able to open adapter com.ibm-df1000fd-1.
121210-202027: ANR9999D_3915179368 StartUtilityModule(psdiscvr.c:1042) Thread<42>: Fork failed
121210-202027: for SAN Utility Module, errno 12
121210-202027: ANR9999D Thread<42> issued message 9999 from:
121210-202027: ANR9999D Thread<42> 0x000000010000c7e8 StdPutText
121210-202027: ANR9999D Thread<42> 0x000000010000fb90 OutDiagToCons
121210-202027: ANR9999D Thread<42> 0x000000010000a2d0 outDiagfExt
121210-202027: ANR9999D Thread<42> 0x0000000100124138 StartUtilityModule
121210-202027: ANR9999D Thread<42> 0x0000000100124c60 psOpenFCAdpt
121210-202027: ANR9999D Thread<42> 0x0000000100282320 naDiscoverDevices
121210-202027: ANR9999D Thread<42> 0x0000000100281b1c NaDiscoverThread
121210-202027: ANR9999D Thread<42> 0x00000001000151b0 StartThread
121210-202027: ANR9999D_3612203376 psOpenFCAdpt(psdiscvr.c:426) Thread<42>: Unable to start
121210-202027: SAN discovery utility module
121210-202027: ANR9999D Thread<42> issued message 9999 from:
121210-202027: ANR9999D Thread<42> 0x000000010000c7e8 StdPutText
121210-202027: ANR9999D Thread<42> 0x000000010000fb90 OutDiagToCons
121210-202027: ANR9999D Thread<42> 0x000000010000a2d0 outDiagfExt
121210-202027: ANR9999D Thread<42> 0x0000000100124c88 psOpenFCAdpt
121210-202027: ANR9999D Thread<42> 0x0000000100282320 naDiscoverDevices
121210-202027: ANR9999D Thread<42> 0x0000000100281b1c NaDiscoverThread
121210-202027: ANR9999D Thread<42> 0x00000001000151b0 StartThread
121210-202027: ANR1787W Not able to open adapter com.ibm-df1000fd-2.
121210-202028: ANR9999D_3915179368 StartUtilityModule(psdiscvr.c:1042) Thread<42>: Fork failed
121210-202028: for SAN Utility Module, errno 12

[tarunfeb27]

Please help as instance is not coming up due to this error

[Harry_Redl]

Hi,

is it a new instance? Has it ever worked?
What about resource limits (ulimit -d unlimited)? "SanDiscovery OFF" in the dsmserv.opt

Just guessing ....

Harry

[tarunfeb27]

It's an old instance, recently i have started facing this problem. 'ulimit -d unlimited' is there.As this instance is library client and library manager is on other AIX instance.
I got few weird thing that the amount of "ANR0408I Session NN started for server <Name> ... for library sharing" is just enormous.
I can see more than 32000 messages per hour in actlog.
Can you please help?

[Harry_Redl]

Hi,

weird - can you see the devices in AIX? Are the devices operational? Are you root?
Seems your server tries to start, finds out it is a library client and tries to initialize the library (contacts the LManager) - but cannot fork threads for SAN operations - so it fails and starts another attempt - can be device problem, permission problem, memory problem ....
A lot of things has to be checked here ...

Harry

[tarunfeb27]

Hi ,
Sorry for late reply
Thank you for your suggestion
I had a deep investigation for the problem and found the both problem are not related to each other.
Multiple session to LM is caused by many select command ,those were running on LC server and asking for volume information to LM(each volume)
However san issue is related to some missing configuration at OS level.I'm taking a hold of it.
LM= Library Manager
LC = Li