We have several v5.1 master servers with 1-2TB catalog sizes (binary / 
compressed).

What we do is set the DB catalog backup to backup:
/usr/openv/volmgr/database
/usr/openv/var
/usr/openv/netbackup/db/images/<master>

..so that the DB catalog backup only backs up the images for itself.

Then create a policy that backs up /usr/openv/netbackup/db/images since that is 
where the majority of the data is.

We like to split it up so the backup selection looks like this:

NEW_STREAM
/usr/openv/netbackup/db/images/[a-g]*
/usr/openv/netbackup/db/images/[A-G]*
NEW_STREAM
/usr/openv/netbackup/db/images/[h-n]*
/usr/openv/netbackup/db/images/[H-N]*
NEW_STREAM
/usr/openv/netbackup/db/images/[o-t]*
/usr/openv/netbackup/db/images/[O-T]*
NEW_STREAM
/usr/openv/netbackup/db/images/[u-z]*
/usr/openv/netbackup/db/images/[U-Z]*


As for catalog consistency checks, definately get support on the phone and ask 
about "nbcc".  You'll have to schedule downtime for the master server to do it, 
and large catalogs can take awhile (especially if they are housed on a lower 
tier of storage).

I would also recommend doing a consulting engagement if you think the problem 
is serious.