The out of memory condition looks like a successful backup, but it really is
not.

How much memory is on the Solaris machine and how many files does the
largest filesystem tree path have?  Unfortunately, there were some memory
leaks fixed in later releases of TSM that may apply all the way back to
3.1.0.8 but they will never be fixed there.

The client you are using is the last one that was created for 2.5.1.  No
vendors support Solaris 2.5.1 any longer.  Sucks, but that is just a
financial fact.  The vendors cannot support every release of an OS anymore.
It is just too costly for them to do certification testing.