What is the file structure that you are backing up? Are there a LARGE amount
of small files the client needs to process?  What is the copy serialization
set at for your copy groups.  If you are backing up active files and the
copy serialization is set to make multiple attempts on an active file,
possibly that can add alot of time.

Just some wild guesses.