Can't backup very large file. DIAG: sessSendVerb: Error sending Verb, rc: -50

[mrl]

I'm unable to backup large files, i.e. 90G or more. This is occurring on a client running Ubuntu, on a virtual network. I am able to back such files on a node outside of the virtual network, running the same version of Ubuntu. I'm not sure this the a factor. Below are the error messages. As an aside, Crashplan can successfully backup such large files from the same node. But we went uto se TSM instead, because our university has it's owner TSM server. So backups and restores using TSM would be much faster. Are there any changes on the server that can be tried, in order to see if it would help? Our TSM server managers unfortunately are not very experienced with such problems. Thanks. - Mark

05/28/2021 19:48:21 ANS0361I DIAG: sessSendVerb: Error sending Verb, rc: -50
05/28/2021 19:48:21 ANS1809W A session with the TSM server has been disconnected. An attempt will be made to reestablish the connection.
05/28/2021 19:48:22 ANS1809W A session with the TSM server has been disconnected. An attempt will be made to reestablish the connection.
05/28/2021 19:57:48 ANS0361I DIAG: sessSendVerb: Error sending Verb, rc: -50
05/28/2021 19:57:48 ANS1809W A session with the TSM server has been disconnected. An attempt will be made to reestablish the connection.
05/28/2021 19:57:49 ANS1809W A session with the TSM server has been disconnected. An attempt will be made to reestablish the connection.
05/28/2021 19:58:03 ANS1228E Sending of object '/mnt/archives/xxx' failed.
05/28/2021 19:58:03 ANS8010E An attempt to backup or archive a file has exceed the maximum number of retries.
05/28/2021 19:58:03 ANS1802E Incremental backup of '/mnt/archives' finished with 2 failure(s)

 

[Trident]

Hi,

You probably know that RC-50 is ip connection failure. Looking into that part is important. Do you use DNS or IP as the TCPS? Are there any log entries/errors on the virtual network?

Try setting tcpwindowsize 0 in your client dsm.sys

Look at timeout values on client and server

Do a trace (dsmc i -tracefil=/tmp/log.txt -traceflags=service , and see if you can find any oddities there. You may also do a tcpdump.

A few needed outputs;
versions of client and server
the local dsm.opt/sys
q actlog for the same time frame (looking for weird stuff)
q opt output from the server (timeout settings)

Since you can back up large file outside the virtual network, I think you have to investigate communication from the client to the tsm server.

The server can easily handle large files, even for very old versions. I do not think that the server is the troublesome part here.

 

[mrl]

Thanks for the information. The large files take hours to backup, and it only runs into a couple of -50 errors per hour. But that eventually, is enough to hit the retry limit. tcpwindowsize is already set to be 0 in our dsm.sys file. Do you know if that retry limit can be changed on the server? We are running TSM client version 7, Release 1, Level 8.0 I don't know what the server is running.

The Ubuntu server that the TSM client is running on, is a major data acquisition server. I cannot find any network messages in it's logs. If there was network problem with it, we would have heard about it from our users . In any event, here is one of the -50 errors that was logged in debug file created by the dsmc command you suggested.

05/31/2021 22:49:38.114 [036910] [2516997888] : txncon.cpp (5488): sendIt: rc: -50
05/31/2021 22:49:38.114 [036910] [2516997888] : txncon.cpp (4970): PFtxnListLoop: updating fsIDToCommit from 2 to 2
05/31/2021 22:49:38.114 [036910] [2506503936] : cmlzwcmp.cpp ( 904): OutputCode: increasing numBits to 12
05/31/2021 22:49:38.114 [036910] [2516997888] : dsfifo.cpp ( 315): fifoQinsert(0x24ba4f0): Posting that next object available.
05/31/2021 22:49:38.114 [036910] [2516997888] : dsfifo.cpp ( 321): fifoQinsert(0x24ba4f0): Queue insert of entry 0x7f838004fe40, return rc of 0
05/31/2021 22:49:38.114 [036910] [2516997888] : txncon.cpp (5064): PrivFlush: SendIt returned -50
05/31/2021 22:49:38.114 [036910] [2516997888] : txncon.cpp (5527): findNextState: Got a neg. rc -50(dec) from comm. function. Propagating rc up
05/31/2021 22:49:38.114 [036910] [2516997888] : txncon.cpp (5085): PrivFlush: findNextState returned -50
05/31/2021 22:49:38.114 [036910] [2516997888] : txncon.cpp (4302): PrivFlush: Transaction completed
05/31/2021 22:49:38.114 [036910] [2535884544] : cmlzwcmp.cpp ( 943): ClearStrTable: clearing string table.
05/31/2021 22:49:38.114 [036910] [2516997888] : dsfifo.cpp ( 315): fifoQinsert(0x24ba4f0): Posting that next object available.
05/31/2021 22:49:38.114 [036910] [2516997888] : dsfifo.cpp ( 321): fifoQinsert(0x24ba4f0): Queue insert of entry 0x7f8380039cb0, return rc of 0
05/31/2021 22:49:38.114 [036910] [2535884544] : cmlzwcmp.cpp ( 904): OutputCode: increasing numBits to 10
05/31/2021 22:49:38.114 [036910] [2516997888] : sesscntl.cpp (2579): ANS1809W A session with the TSM server has been disconnected. An attempt will be made to reestablish the connection.

 

[Trident]

Hi,

When you type dsmc, you will see the server version and client version.

RC-50 is something I see too frequently, and there are only a few cases. Which one that applies in your case is difficult to see from the logs provided. I would like to see what happened on the SP server.

- DNS resolver on client if TCPS is a FQDN. Add ip/fqdn to hosts file
- Switch config denies access for if trafic is too high
- Routers drops frames due to hihj load
- Local firewall restricts traffic (iptables?)
- Firewall's between you and the tsm server blocks traffic
- QOS rules
- IP config on tsm server limits traffic

 

[marinaz29]

AFAIR was a problem with session timeout, 15 minutes default isn't enough.