Problem with Domino TDP consumin 100% CPU

[noodles]

Hi,

we have a Domino 8.50 server running on Windows Server 2008 SP2, 64bit on a HP DL 380G5 with two Quad-Core CPUs and 8 GB RAM.
Using nsf-DBs.
We have about 1400 DBs running on it consuming about 1,4 TB.

We have a problem with our selective backup running on sundays.
It consumes 100% CPU on all eight cores and domino server becomes unresponsive!
Sometimes the domino server even crashes.

In domdsmerror.log I just get:
"03/07/2010 11:09:50 ANS1512E Scheduled event 'DOMO8-FULL' failed. Return code = 5"

In domsel.log I sometimes get:
"ANS1017E (RC-50) Session rejected: TCP/IP connection failure" but after a retry the backup of the nsf mostly works.

Actlog just says following:
"ANR0480W Session 621 for node DOM-TDP (TDP Domino WIN64) terminated - connection with client severed. (SESSION: 621)"

The TDP has version 5.5.2.

I have attached some config files and our domsel.cmd.
Help is really appreciated.

regards
/noodles

 

[micheledenardo]

I have the same problem. Maybe the only difference is that the CPU (8 Xeon cores) is normal for several hours, but later spikes to 100% and then the server does not respond any more.
The normal load for the CPU is about 3%.
Does anyone have any suggestion?

 

[terlisimo]

Which process is consuming 100% CPU? I assume it is domino server, not the TDP client... since client should use only one core. Similar thing happened to me when I forcibly terminated a domino backup. It left the service in a half-crashed state... backup was still able to run but domino server was otherwise unresponsive.

Currently I'm backing up 1TB of Domino mailboxes in similar environment (8.5 on 64-bit windows) without problems. Backups complete in 5-6 hours with client disk subsystem as the bottleneck.

I'd suggest upgrading to 5.5.3 which is the latest version of TDP for Domino on Windows.

Maybe give this a try: turn on client compression and use 2 simultaneous sessions for backup... this way each session will transfer only 50% of total data size. Each session should run its own process so you should see 2 cores used by the TDP backup client.

Also, check your log utilization on TSM server during backup... maybe you have problems with long-running transactions. Using multiple sessions should help in this case too.

Finally, you may try to experiment with idletimeout and commtimeout values on the TSM server. Maybe the control session times out while the data sessions is still active (shooting in the dark here).

Bear in mind that using multiple sessions will probably put a greater load on your Domino server's disk subsystem. My Domino databases are on iSCSI storage and the server is barely responsive while the backup is running.