OS X 5.2.3 Client - ANS1030E System ran out of memory

Hi All,

I need some help! I have an OS X Server (10.3.5) node for which the daily incremental backup keeps failing. There are 4 very large XRAID partitions that contain hundreds of thousands of files that I need to backup incrementally every day. The backup always fails with the same message.

ANS1802E Incremental backup of 'haza:*' finished with 1 failure
Total number of objects inspected: 434,049
Total number of objects backed up: 513
Total number of objects updated: 0
Total number of objects rebound: 0
Total number of objects deleted: 0
Total number of objects expired: 200
Total number of objects failed: 1
Total number of bytes transferred: 166.34 MB
Data transfer time: 35.63 sec
Network data transfer rate: 4,779.28 KB/sec
Aggregate data transfer rate: 71.90 KB/sec
Objects compressed by: 0%
Elapsed processing time: 00:39:28
ANS1030E System ran out of memory. Process ended.

I have watched memory utilization during the backup. Here is the typical load while backing up:

PhysMem: 180M wired, 173M active, 647M inactive, 1001M used, 1.02G free

Disk space on the root volume is fine too. So if the system does page it has plenty of free space in /private/var/.

I have manipulated all the TSM tuneables with no success. Here is my system options file:

servername hamm
WEBPORTS 1000 0
nodename madrid
SCHEDMODe prompted
COMMmethod TCPip
TCPPort 1500
TCPServeraddress hamm.corp.apple.com
TCPBuffsize 64
TCPWindowsize 64
Compression Off
TCPNodelay Yes
LARGECOMmbuffers Yes
NFSTIMEout 120
TXNBytelimit 25600
memoryefficientbackup yes
ERRORLOGRetention 5d
ERRORLOGName /ngs/lpp/dsmerror.log
SCHEDLOGRetention 2d
SCHEDLOGName /ngs/lpp/dsmsched.log
INCLEXCL :Volumes:ngs:lpp:inclexcl.tsm
PASSWORDACCESS generate

No errors in the hardware, crashreporter, or system log. The frustrating thing is I have three other identical system that don't have the problem. Any help or suggestions would be greatly appreciated.

thanks,

Greg