BackupPC-users

Re: [BackupPC-users] suggestion: make wakeups start at 8am

2009-07-08 22:12:50
Subject: Re: [BackupPC-users] suggestion: make wakeups start at 8am
From: Adam Goryachev <mailinglists AT websitemanagers.com DOT au>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Thu, 09 Jul 2009 12:07:11 +1000
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Holger Parplies wrote:
> Hi,
> 
> Adam Goryachev wrote on 2009-07-09 09:24:05 +1000 [Re: [BackupPC-users] 
> suggestion: make wakeups start at 8am]:
>> Filipe Brandenburger wrote:
>>> On Wed, Jul 8, 2009 at 11:39, Les Mikesell<lesmikesell AT gmail DOT com> 
>>> wrote:
>>>>> Also, as an aside, the process should probably be renamed to
>>>>> BackupPC_daily, as it makes more sense to run it during the day when
>>>>> backups are not running. Isn't that right?
> 
> I don't think the name is important. It's an implementation detail. You can
> change related config settings and the documentation without renaming the
> binary. Does anybody care about the name "BackupPC_Admin"? People should read
> documentation, and if they are about to change the time BackupPC_nightly is
> run, they almost certainly have. I would guess that the origins of the name
> are probably from the observation that most maintenance tasks are run during
> the night, when they don't disturb anyone. Server backups are an example of
> such a task, as are "daily" cron jobs.

The name only matters in that people may infer meaning where it is not
meant. Though this could be fixed through documentation, and a few
interface changes. For example, if the web interface config displayed
the setting as DailyMaintenance instead of NightlyMaintenance even
though the binary that uses the setting was Backuppc_nightly... If
someone knows enough about backuppc to find the binary name, then they
should also know enough (to find out) what it does and why.

> Coincidentally, this maintenance task is not supposed to disturb backups ...

Except it is pretty well acknowledged that it does due to the high
amount of disk IO... Of course, there are things that could be done to
off-set that, such as a small sleep every 100 "bits" it processes, but
it will *still* impact on backup performance, just less...

> But you should keep in mind that not everyone has a dedicated backup server.
> For example, I run BackupPC on an NFS/NIS/whatever-server to an external iSCSI
> device. The server is idle during the night, so why not use it as BackupPC
> server? The storage unit is independent; that is what seems to matter.
> Obviously, I prefer BackupPC_nightly disturbing the backups rather than my
> users :). I *have* seen it run during the day, and it wasn't being very subtle
> about it.

Exactly, and that is how obviously it will impact on backup performance
as well.

> If BackupPC_nightly is renamed, then why not to something that makes sense,
> like BackupPC_dailyMaintenance or BackupPC_poolMaintenance or just
> BackupPC_maintenance?

Keep in mind that is is also responsible for sending out the reminder
emails, updating the status.pl, and probably other stuff I don't know
about... It doesn't *just* maintain the pool...

> Agreed. The default should be good enough for most people - particularly
> unexperienced users -, disruptive for nobody, and follow the principle of
> least surprise. Early morning seems reasonable. Inside the blackout window
> makes sense but may be surprising.

Only in that a user may think the blackout period applies to all
backuppc activity as opposed to backup hosts activity... However, I
agree that it should be scheduled to occur (by default) during the
(default) blackout period.

>>> As you said yourself in another thread, even though BackupPC_nightly
>>> can be run while other backups are running, it is better to do so
>>> while no or few backups are running so that they do not compete for
>>> disk I/O
> 
> Which is why I'd argue for making BackupPC_nightly run time independent of
> the WakeupSchedule. The WakeupSchedule defines possible times when backups
> may be started (though they may continue or be started delayed due to
> MaxBackups). Having an independent time for BackupPC_nightly where no backups
> are considered makes sense, eg.
> 
>       $Conf {WakeupSchedule} = [ 21, 22, 23, 0, 1, 2, 3 ];
>       $Conf {NightlySchedule} = [ 7.75 ];

Not a bad idea, for example if I know all my hosts have a blackout time
of 7am - 7pm, then why do I need backuppc to wake up and schedule all
the hosts, only to find out that backups are not needed? However, I do
need it to wake up and process the pool at 10am...

Also keep in mind that unless you have a very small pool, and/or very
fast (disks) pool, then your daily maintenance will probably take much
more than 15 mins... (Mine takes around 3 hours).

>> I also agree that setting the default time inside the default backup
>> window is a good idea,
> "Backup window" or "blackout window"?

Damn, did I really say that :( I of course meant:
I also agree that setting the default time inside the default blackout
window is a good idea,

>> that way if someone decides to change the
>> default, then it is up to them to also change the other value...
>>
>> I also agree on renaming to daily, consider /etc/cron.daily etc, it is
>> something that should be run every 24 hours, not needed to run at night.
> 
> "Daily" is not strictly a requirement. "Regularly" is (but BackupPC_regularly
> is a bad name ;-). BackupPCNightlyPeriod effectively makes it traverse the
> pool less often, and that's ok (hmm, what about e-mails, though?).

Well, we could hive off the pool maintenance into a new binary called
backuppc_poolmaintenance and have that run separate from the
backuppc_dailymaintenance which does the email, status.pl updates,
etc... In fact, if that happened, backuppc_dailymaintenance could become
backuppc_maintenance or similar, and could run more than once a day,
since it is so low impact... As long as it knows not to send more than
one email per day... Assuming this has any advantage to anyone to run it
more than once daily (or less...)

>> As to running it at intervals less than 24 hours, I'm not sure I see
>> the advantage, better to just get it to process a smaller section of
>> your backup pool each day, rather than run it on the whole pool every
>> x days...
> 
> I could imagine reasons to want a complete pool traversal on a particular day
> rather than splitting it up equally. You might need to run full backups of all
> your servers over the weekend and have no time left for a _nightly. Or you
> can't run any backups on Wednesdays for some obscure reason, so that's the
> ideal moment for pool maintenance. Probably not very common. If it fits into
> the design, why not make it possible - someone will find a use for it -, if it
> doesn't (as currently), then don't bother - nobody will miss it.

True, and this could be done better if backuppc_poolmaintenance was a
different process. In fact, again it would make it possible to disable
pool maintenance within backuppc, and schedule it from cron to give a
totally flexible schedule (ie, every wednesday in the first week of the
month at 3am, or whatever)...

I guess what we all should also recognise is that we don't do loads of
work where none is really required. The simplest change (ie, change of
default value, and changing the documentation) should be done, the rest
of the suggestions being discussed are much more optional (IMHO).

Regards,
Adam

- --
Adam Goryachev
Website Managers
www.websitemanagers.com.au
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkpVUMsACgkQGyoxogrTyiXSEQCfRtIOVesI8MHvWcWI6iPyq0Fe
lJ4AoJwwXXtKVX3vLVbuTN2R3BBdnhM0
=7bvU
-----END PGP SIGNATURE-----

------------------------------------------------------------------------------
Enter the BlackBerry Developer Challenge  
This is your chance to win up to $100,000 in prizes! For a limited time, 
vendors submitting new applications to BlackBerry App World(TM) will have
the opportunity to enter the BlackBerry Developer Challenge. See full prize  
details at: http://p.sf.net/sfu/Challenge
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/