Summary: "The sky is falling!", oh, and IBM, when do we get a
remote-storage-service DEVTYPE?  We're gonna need it, and how.



So I'm wandering around chasing some links in Amazon's web-services
space.  I stumbled across S3, their storage service.  What drew my
mind particularly to it was that their charging structure is identical
to my TSM chargeback: one term for storage, and another for transfer.

They're charging 20 cents per GB of transfer, and 15 cents per
GB*month of storage.  So they're 1/3 my cost for transfer, and 5* my
cost for storage.

Not an immediate threat, even with trivial analysis.  But dang, that's
an uncomfortable bracket.


So what are the other factors?

+ Security of data. If someone comes up with a reasonable encryption
  layer to slap onto the web-services abstraction (and I'm betting
  dozens of stabs at this already exist) this is a non-issue.   Some
  organizations will be simply unconvincable about this, others will
  find it a no-brainer.

  Once someone gets HIPPA, or FIRPA validation for an encryption
  scheme intended to cover media you give to third parties, the rest
  of them will fall.

  For some shops, the local solution already loses on this point.
  Other converts will happen on an undecidable timeframe, modified by
  regulatory nonsense.  (double-undecidable?)


+ Lots of bandwidth to [wherever Amazon is].  That doesn't sound like
  too much of an obstacle.   They're already doing bittorrent access
  to objects; it seems plausible that bandwidth to their service will
  rapidly tend towards "how big is your internet pipe".

  We've all got more internal bandwidth than external, but that's
  handwaving.  Getting a 1G or a 10G connection between your server
  farm and The World is just a fiscal question.

  I still win this hands down, but on a timescale of years, single
  digit.


+ Management interface.  If you're using this abstract storage for
  backup, where's your "TSM database", the directory to tell you what
  data is interesting and what's not?

  That's going to end up with many solutions imitating the TSM
  landscape: local database of where stuff is, reclamation of volumes
  when estimated future-cost-of-storing-emptyness crosses
  cost-of-copying, etc.

  There are three entries in the S3 "solutions catalog" which
  explicitly talk backup. This represents a lot of work, but again
  there are probably shops without experience of TSM's organizational
  tools which are being captured now.

  I win now, but there are suddenly lots of hungry competitors
  watching what we do, and learning.  I don't think TSM can compete
  against the Open Source gestalt if reliable storage is cheap
  everywhere.


+ Risk of media loss.  What if the Amazon service loses your data?
  really hard to evaluate that one.  Like encryption, I can see some
  shops shrugging, and some recoiling.  More likely than "they lose
  it" is "They can't get it to you quickly".  I expect this service
  level, in reality, to approach perfection, though I know some
  administrators for whom that won't ever matter.   In engineering
  terms though, it is probably already Good Enough for most of our
  needs. (they claim 4 9s and can probably back it up)


+ Latency.  S3 will blow me away unless I go to all-disk.  This is a
  no-brainer. For user-level restores, the S3-backed storage will be
  overwhelmingly more responsive than anything I can do at that
  scale.

  Local (tape) solution loses hands down.


+ Maintenance.  Local solutions lose overwhelmingly. BIG time. I just
  send my bits to tbe Big Place in the Sky. No device type issues.

  Super lightweight client: Want to do a restore on a new box?
  install the code, tell it who it is.  It authenticates to Amazon,
  downloads the map of where the database is.

  Downloads the database.

  Downloads the data.

  I lose again.


IBM, if you're not thinking of this as a shot off your bow, both in
storage hardware and in TSM, then we're sunk.  Google's going to do
something like this too, and I understand M$ is on board.
"LiveDrive"?

The price is going to keep doing the better-than-moores' slump that
magnetic media have been exhibiting for the last decade or two.

When their prices are lower than mine on the face of it, then I have
to start explaining.  When they're half what mine are on the face,
then the explanations won't help.



- Allen S. Rout