Amanda-Users

Re: S3 Backup using 2.6.1b2

2009-01-12 12:28:57
Subject: Re: S3 Backup using 2.6.1b2
From: Graham Wooden <graham AT g-rock DOT net>
To: "Dustin J. Mitchell" <dustin AT zmanda DOT com>
Date: Mon, 12 Jan 2009 12:23:10 -0500

Also keep in mind the TCP sliding window when going over the Internet. You can have a fat pipe (on both ends), but the increased hop count and the TCP overhead you will only get a percentage of your pipe. In my experiences, they have been about less than 1/2 of the pipe. Unless the "backup" is streamed over UDP (more efficient but no error checking).

My two cents.

-graham

Quoting "Dustin J. Mitchell" <dustin AT zmanda DOT com>:

Sorry for the delay here.

On Thu, Jan 8, 2009 at 11:32 AM, Matt Burkhardt <mlb AT imparisystems DOT com> wrote:
I'm not too sure what's going on - it takes about 10 days to run the
backup.  Here's the results from the run that started on 12/27 and the
results from amstatus on the backup that started running yesterday.  I
checked S3, and it used up a little over 5000 "tapes".  Is there a time /
amount limit?

5000 tapes?  But your tapecycle is only 14.. do you mean 5000 S3 objects?

You're getting ~60k/s.  That's a little slow -- I get 115k/s on my
home system (Comcast) and 179k/s on a business DSL in Cleveland -- but
even tripling your speed will still leave you taking ~100h to upload
56G.  S3 backups are more appropriate to data sizes around 1-2G/night
(about a 7 hour backup window at your speed).  There's just no good
way to upload 56G "quickly," unless you have a very fat pipe[1].

Dustin

[1] And my experiments on such a fat pipe show that Amazon has some
kind of rate limits, too, so even this isn't a great solution.  My
tests were done two years ago, so I'd be curious to know if this is
still true.

--
Storage Software Engineer
http://www.zmanda.com





<Prev in Thread] Current Thread [Next in Thread>