I've been using bacula with S3 using s3fs for about 6 months. To say
it has been flakey is an understatement and I plan to make changes
over the coming months, although still using s3/glacier. s3fs is a
filesystem for S3, I believe the only free one (it's GPLv2).
Backups:
Small backups are fine, but large backups taking > ~2 days inevitably
have problems (ie connection drops) and neither s3fs or bacula are
good at resuming transfers/backups. Large backups need to be stored
locally and then transfered to s3 post job.
It's also extra important to backup the database straight after the
initial full backup as if something goes wrong you'll have to spend a
long time re-upload/downloading full backups.
Restores:
Restoring directly using s3fs is only possible if the volume is
completely cached locally by s3fs. I've never managed a backup that
hasn't had the entire volume manually downloaded (either filling the
cache or just cheating and remounting), so using a small volume size
is essentially. I look at the volumes required when using the restore
command and manually get them to cache.
My Future Plan:
Ditch s3fs, it's not reliable enough. Instead I plan to use a
combination of s3cmd and a simple post job script to verify correct
transfer. s3cmd is just a cp/mv/rsync kind of tool and so doesn't
suffer of the issues s3fs does. My only concern is correctly verifying
transfers have been successful.
I've never used AWS storage gateway, the economics don't work in my
instance, I run multiple small sites - it's $125 a head. This might be
an option for some, but I still think baculas inability to resume
backups would be a problem.
Perhaps if bacula had a built cache, copy and verify mechanism use of
such offsite backup services would be easier and safer. I.e. do backup
to local disk (to avoid large jobs taking weeks/months and potentially
bombing half way through), copy these volumes to another location and
then verify the copy. A similar procedure in reverse would also remove
some headaches for restorations.
Edward
On 21 August 2012 15:39, Mr IT Guru <misteritguru AT gmx DOT com> wrote:
>
> Good Afternoon All,
>
> On 21 Aug 2012, at 13:38, eric santelices <ericsante AT gmail DOT com> wrote:
>
> With Amazon announcement of Glacier it seems like a natural fit for Bacula.
> Does anyone know if/when we would see support for this storage resource?
>
>
>
> If you wanted to utilize this, wouldn't it be better to have the OS look into
> mounting this service as a file system, and then just direct bacula to use
> the file system? Rather than coding specifically for bacula - I'm sure bacula
> is not going to be the only project that can benefit from this, so I'm
> betting that support for this will be in the underlying OS first.
>
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Bacula-users mailing list
> Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
|
