On 05/30/2013 02:51 PM, Ehresman,David E. wrote:

> We use Docave.  IBM used to resale this as TSM for Sharepoint.  They
> are no longer doing that but you can still get the same functionality
> directly from Docave.


M.o.l.a.s.s.e.s.

Seriously.

When it takes hours and hours and hours AND HOURS, that's not a bug,
that's how it works.

As far as I can tell, it goes like this:

They started with a big website.

To be efficient, they wanted it inside of a database, so they put it in
a database.

They wanted to keep lots and lots of documents in the website
(database), so they put blobs in the database.

Lots of blobs are hard to organize, so they wanted a familliar
organizational schema.  People are familiar with a filesystem, so they
made it look sort of like a file system.

Sharepoint is well optimized for seamless integration with the Win
desktop, and that mode of utilization is explicitly encouraged in lots
of places, so e.g. your project notes and stuff all go in "website"
directories.  Including rudimentary version control.  So the blob count
in the database skyrockets.

access control metadata ramifies, so you've got a metadata vocabulary
that is more complex and nuanced than just the NTFS universe:  your
access control includes opinions about the outside world, too.  (mostly
"NO!", but you've got to write that down.).


... All of this may be semi-obvious, but the point of restating it is to
bring to the forefront of your mind that, from the back-end view,
Sharepoint has a great deal in common with a complex, customized,
locally-hacked-upon filesystem implementation.

So, what DocAve does, is it walks that filesystem and does an
incremental.  But instead of reading something optimized, down to the
block level, for rapid access, it's doing a series of database accesses.

Think about how long ls -l would take if every file metadata read
required two or three index scans and their indicated block retrievals?
 Ick.


... Our sharepoint guy, Joe Gasper (Hi, Joe!)  thinks I'm not talking
too much bullshit with this description, and he adds that exporting the
BLOBs is a critical performance enhancement for DR, and substantial for
normal access, too.  Well tuned, he thinks 95% reduction in the SQL
database size is readily achievable.  At that rate, you can fit the
remaining 5% on SSD...


- Allen S. Rout