Resource Locks.

[shcart]

Anyone else have similar issues. We are running SP v8.1.12.120. I have just had to bounce our SP server for the second time in 2 days because of resource locks.

We have a server with tdp4ve clients that ran for months with no Issues. A couple of months ago, No changes on server or Client we started getting excessive failures as we passed about 85% DB Occupancy on the way down from Mid/High 90%'s. It appeared the client was just stopping talking to the server, we initially suspected a network issue. IBM recommended increasing resourcetimeout from 60 to 180 which we did. Now 2 months later and no client or server changes at all we are starting to get resources hanging when expiration and remediation backups are running and competing for resources.

DB 11.9 TB, 2.8TB free, Dir Pool is 760TB 28% Occupancy. Processes running Expiration, Running Audit License, about 50 VM Backups over 2 AsNodes. (~150 Session)

 

[marclant]

I think it's time you consider splitting that server instance in 2 (possibly 3). The maximum supported size for the database is 8 TB (https://www.ibm.com/docs/en/spectrum-protect/8.1.14?topic=planning-database-space-requirements). This limit is not enforced by the software, but it's a practical limit. Once it exceeds that, the database backup takes longer because it's larger. Expiration also takes longer to run. So housekeeping processes and client backups start to overlap more and more. And when there's a lot of overlaps, the risk of resource lock contention increases as you are seeing.

When expiration is running against a filespace, there is some locking on the database records for that filespace, so if a client backup is trying to update that same filespace at the same time, it will need to wait until the lock is released or the resource timeout is exceeded, whichever comes first. You could extend the resource timeout, but that's just band aid because it means the backups may take longer to finish instead of failing early. Neither is desirable, the goal is to finish the backups successfully quicker.

The best you can do short term is tuning the database optimally if it is not already following the large Blueprint. But it's not likely to solve your issue completely, but just give you some improvements.