Bacula-users

Re: [Bacula-users] Sending tapes off-site

2008-09-23 11:31:48
Subject: Re: [Bacula-users] Sending tapes off-site
From: "Robb Wagoner" <robb AT dcexp DOT net>
To: bacula-users AT lists.sourceforge DOT net
Date: Tue, 23 Sep 2008 08:29:00 -0700
> Message: 11
> Date: Mon, 22 Sep 2008 22:23:46 +0200
> From: Arno Lehmann <al AT its-lehmann DOT de>
> Subject: Re: [Bacula-users] Sending tapes Off-site
> To: bacula-users AT lists.sourceforge DOT net
> Message-ID: <48D7FED2.8050904 AT its-lehmann DOT de>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> Hello,
>
> 22.09.2008 21:38, Robb Wagoner wrote:
>> I want to send the monthly full backups off-site for DR purposes. I
>> also need to keep an on-site copy of the full backups on-site in the
>> event of non-DR restores.
>
> Good idea.
>
>> My thoughts are to use either:
>>
>> 1. bcopy. Keeping track of  the tapes that are used for full backups
>> and which of those have been duplicated.
>
> bcopy can be a bit trick IMO. It's not as thoroughly tested as the
> core of Bacula (no automatic regression testing, as far as I know). I
> never really used it, though.
>

bcopy /seems/ to work ok. I haven't done exhaustive testing or trials
using it. I think the tricky/complicated part is the management of the
overall copying process. Tracking which tapes are fulls and need
duplication, which of those have completed duplication, watching for
and handling errors, etc.

>> 2. Migration. Full backups are written to an Off-Site Pool and
>> migrated to an On-Site Pool.
>
> The disadvantage is that a migrated job is only present on the target
> volume as far as Bacula is concerned. The actual data is not deleted,
> but it's no longer useable for restores without some catalog tweaking.
>

That has been my understanding. Presumably, in the event of a DR
scenario where the offsite tapes are needed, a bscan would be
necessary. But that is an assumption on my part. I haven't verified
that results in the expected behavior.

>> I don't have adequate disk space to make one of the pools file-based,
>> which means that for either 1., or 2., I won't be able to run any
>> other jobs while the duplicating process (Migration or bcopy) is
>> running as both tape drives will be in use.
>>
>> My inclination is to use Migration:
>>
>> * Pool "Offsite" with 6 month retention period; "Volume Use Duration"
>> of a several days so that the jobs on a partially used tape are
>> migrated; "Next Pool" = "Onsite"
>> * Pool "Onsite" with 1 month retention period
>> * Full backup jobs use pool "Offsite"
>> * Migrate jobs from "Offsite" to "Onsite"
>> * Set Migration job priority lower than backup jobs so that they are
>> effectively done when nothing else is going on.
>
> Sounds reasonable, basically.
>
>>  I will then have to create a custom SQL query (query.sql) to
>> determine which tapes that are in pool "Offsite" & in the library &
>> have been fully migrated. These are the tapes that are ready to be
>> sent off-site.
>
> The query itself shouldn't be too complicated.

Agreed.

>
>>
>>
>> I am wondering what others do for this. . .
>
> Well, until job cpies are fully implemented in Bacula, what I tend to
> do - or rather, advise to do - is doing full backups twice. The
> downside is that this doubles the load on the machines backed up, and
> it can take too long for your backup windows.
>
> The advantage is that you can use the jobs for restores without any
> tweaking.

I don't participate in the bacula-devel, and "job copies" already be
discussed and planned.
To your knowledge, Arno, are "job copies" currently on the known
Bacula "roadmap"?

> Just keep in mind that, if you use the same job setup, you must run
> the offsite backups before the onsite ones, so that later differential
> and incremental jobs will reference the onsite job. Otherwise, you'll
> regularly need the offsite volumes for restores...

Understood.

> Sample schedule:
> - Full job for offsite on Friday 22:00, pool offsite
> - Full job for onsite on Saturday 12:00, pool onsite
> - differential / incremental jobs for onsite during the weekdays, pool
> onsite. These jobs will refer to the latest full job, i.e. the onsite
> ones.
>
> You can then, on Monday, just take all the used offsite pool volumes
> and ship them.

I like this approach for the reasons you've enumerated. Unfortunately,
fulls are taking about 3 days, with one client in particular taking
the longest (x2 NFS mounted NetApp volumes).
I think I'll rework how the NetApp is backed up (one job per volume
vs. one job for both volumes). That might shrink the backup window,
albeit slightly, as well as making the two-full-backups approach
workable.

>
> Arno

Thanks for your feedback Arno.

-Robb

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

<Prev in Thread] Current Thread [Next in Thread>