BackupPC-users

Re: [BackupPC-users] Different Blackout Periods

2015-04-01 17:41:26
Subject: Re: [BackupPC-users] Different Blackout Periods
From: Holger Parplies <wbppc AT parplies DOT de>
To: Les Mikesell <lesmikesell AT gmail DOT com>
Date: Wed, 1 Apr 2015 23:37:45 +0200
Hi,

Les Mikesell wrote on 2015-04-01 13:25:56 -0500 [Re: [BackupPC-users] Different 
Blackout Periods]:
> On Wed, Apr 1, 2015 at 1:03 PM,  <alexander.rehbein AT fmex DOT de> wrote:
> >
> > I want to setup the following backup plan. Every week one full backup
> > and every hour an incremental backup.
> >
> > This works but i have a problem with the balackout periods. Full backups
> > should only done at night. is it possible to configure this?
> 
> There is no way to specify different blackouts for fulls vs.
> incrementals.

true.

> However fulls won't happen until the FullPeriod time
> has expired since the last one, so if you force one at an appropriate
> time, subsequent runs will happen at about the same time a week later.

That is not strictly true. If I remember correctly, a new full backup will not
happen until the FullPeriod has expired since *completion* of the last one
which might be significantly later, depending on the time the backup takes and
backup concurrency issues. Usually, you can compensate for that with the
schedule, blackouts, and an appropriately lower FullPeriod, to keep your
backups from wandering through the day.

Here, however, blackouts can't be used. What you suggest is very much
equivalent to "implement the effect of blackouts by just forgetting about
them". If blackouts are a requirement, i.e. full backups *must* only be done
at night, rather than "it would be rather convenient if they mostly run at
night until something unexpected happens, when we will be happy to manually
intervene to correct the timing", then this is not a solution.

I've got a small patch which I would have first expected would do what you
want, however the task is not as trivial as it seems. BackupPC schedules an
incremental backup or a full one as appropriate. When a full one is due, it
is rescheduled until it completes successfully, perhaps being skipped several
times due to blackouts. In the mean time, there is no point in attempting
incremental backups, because the same blackouts would apply. What you really
want to happen when a full run is skipped due to a blackout period applying
only to full backups is an incremental backup to be run instead. However, a
manually scheduled full backup should probably not be automatically turned
into an incremental. Overlapping blackout periods for full and incremental
backups should work as expected, i.e. an incremental replacement backup should
still be restricted by blackouts for incrementals, regardless of the order in
which blackouts are specified. So there are some things to keep in mind.

An additional problem is that, programmatically, BackupPC currently handles
blackouts *before* making the decision whether a full or incremental backup
is needed. There may be a good reason for that (such as avoiding pings during
blackouts - pingCmd might attempt to wake up the remote host) or not, I'd
have to take a closer look. In any case, it would be significantly more
intrusive than changing two lines of code as my first attempt would have.

Configuration wise, I would want to add an optional hash member backupType
with supported values 'full', 'incr' or 'all', defaulting to 'all', which
would make the blackout period apply to backups of the selected type(s), e.g.:

        $Conf{BlackoutPeriods} = [
                {
                        'weekDays' => [ 1 .. 7 ],
                        'hourBegin' => 4,
                        'hourEnd' => 2,
                        'backupType' => 'full',
                },
                {
                        'weekDays' => [ 1 .. 7 ],
                        'hourBegin' => 2,
                        'hourEnd' => 4,
                        'backupType' => 'incr',
                },
                {
                        'weekDays' => [ 3 ],
                        'hourBegin' => 1,
                        'hourEnd' => 3,
                },
        ];

(meaning no full backups from 04:00 to 02:00, no incremental backups from
02:00 to 04:00, no backups at all on wednesdays from 01:00 to 03:00, just to
give an example).

Ah, right, the web GUI would need to support setting this value, too :-).


So, while I can appreciate the need for this feature and don't see a way to
emulate it with current BackupPC capabilities, I can't offer you a solution
right now. I would suggest trying something like triggering full backups with
a cron job (and raising FullPeriod accordingly to avoid interfering automatic
fulls) for now, though this is obviously only an approximation, too.


You do realize that with your configuration the hourly incrementals will
be growing deltas to the previous weekly full backup, right? That may not be a
problem, but with hourly backups you probably need these to be really fast. Is
retransmitting the same changes over and over again going to work for you?

Hope this helps.

Regards,
Holger

------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the 
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/