Re: S3 Backup using 2.6.1b2
2009-01-12 12:28:57
Also keep in mind the TCP sliding window when going over the Internet.
You can have a fat pipe (on both ends), but the increased hop count
and the TCP overhead you will only get a percentage of your pipe. In
my experiences, they have been about less than 1/2 of the pipe.
Unless the "backup" is streamed over UDP (more efficient but no error
checking).
My two cents.
-graham
Quoting "Dustin J. Mitchell" <dustin AT zmanda DOT com>:
Sorry for the delay here.
On Thu, Jan 8, 2009 at 11:32 AM, Matt Burkhardt
<mlb AT imparisystems DOT com> wrote:
I'm not too sure what's going on - it takes about 10 days to run the
backup. Here's the results from the run that started on 12/27 and the
results from amstatus on the backup that started running yesterday. I
checked S3, and it used up a little over 5000 "tapes". Is there a time /
amount limit?
5000 tapes? But your tapecycle is only 14.. do you mean 5000 S3 objects?
You're getting ~60k/s. That's a little slow -- I get 115k/s on my
home system (Comcast) and 179k/s on a business DSL in Cleveland -- but
even tripling your speed will still leave you taking ~100h to upload
56G. S3 backups are more appropriate to data sizes around 1-2G/night
(about a 7 hour backup window at your speed). There's just no good
way to upload 56G "quickly," unless you have a very fat pipe[1].
Dustin
[1] And my experiments on such a fat pipe show that Amazon has some
kind of rate limits, too, so even this isn't a great solution. My
tests were done two years ago, so I'd be curious to know if this is
still true.
--
Storage Software Engineer
http://www.zmanda.com
|
|
|