> Anyone have a particularly large HSM environment and been using it a while?

We have been using ADSM HSM for years, principally for archiving system log 
files
and like stuff, which we retain for five years.  HSM makes this practical, and
affords obviousness of the stored data (unlike ADSM Archive) and ready 
retrievability.
Suffice to say we have many gigabytes of it.  For this it works nicely.

We had early on anticipated rolling out HSM more generally - but then realized
the problems.  HSM is a DISASTER when used over NFS...  Picture gigabytes of 
data
flowing into a file system over NFS; then the file system fills.  Automigration
makes what space it can - but that's not enough.  Reconciliation is now running
on the file system, but that can take up to 2 hours.  Now the file system is
full, and the HSM session is wedged.  This causes *all* NFS service from that
system to cease, and wide-spread chaos ensues as even innocent 'df' commands
issued on other systems which have that file system NFS-mounted hang, and can
not be freed until the NFS server again responds.  This, in combination with
fragmentation issues and other factors, caused us to deem HSM not for civilians.
HSM has its uses, but is awfully Rube Goldberg.  You may first want to consider
file systems which autocompress data stored in them.

  Richard Sims, BU