BackupPC-users

Re: [BackupPC-users] backup of backuppc and schedule, is it archive?

2010-03-12 04:28:08
Subject: Re: [BackupPC-users] backup of backuppc and schedule, is it archive?
From: Sylvain Viart - Gmail <sylvain.viart AT gmail DOT com>
To: backuppc-users AT lists.sourceforge DOT net
Date: Fri, 12 Mar 2010 10:26:02 +0100
Hi Les,

On 11/03/2010 18:05, Les Mikesell wrote:
Backing up some primaries backuppc servers, on a secondary backuppc.
One of the primary hold itself 86 host.
    
Are you trying to back up the primary backuppc servers complete with 
history, or do you just need the latest full from each target in this copy?
  
Just the last full.

Following your advice, I've changed my strategy to provide a quicker way to restore any host data, directly from the secondary.
Backuping /var/local/machine, I've all final host data living there, on the secondary.
This folder in maintained by my BackupPC_tarCreate loop.

The secondary backuppc provides the following:
  • external backup, they are off site on another datacenter
  • some primary host are also giving service other than backuppc, and should be secured in case of data lost
  • secondary backup should be never used, just in case of serious damage
  • no config sync, nor schedule sync is needed between secondary and primary backuppc
  • don't need the history of each target (final host) on the secondary, only an up to date last backup

Here's how it looks like on the secondary, is it not somewhat a similar tree that is living somewhere in backuppc?

machine/
|-- server-01
|   |-- etc
|   `-- root
|-- server-02
|   |-- etc
|   `-- root
|-- server-03
|   |-- etc
|   `-- root
|-- other-10
|   |-- etc
|   |-- root
|   `-- var
|-- other-11
|   |-- etc
|   |-- root
|   `-- var
`-- moreserver-01
    |-- etc
    `-- root


Strategy:
Using a BackupPC_tarCreate loop on the primary, called from a
DumpPreUserCmd on the secondary backup server.
Then the secondary backup itself the copy of every extracted host.
    
This seems like you are adding a bottleneck compared to just backing up 
the targets directly from the primary and secondary backuppc servers.
  

Backup of final host twice should be avoided, because:
  • need to keep the config in sync between, primary and secondary backup server.
  • the backup job is too heavy for the final host. I prefer to use the copy which lives on the primary backup.
What is the bottleneck?

If you hit the hosts directly you could use rsync.  Maybe you could use 
the same ssh identity key and just script the config updates to 
propagate changes as you add hosts to the primaries.

  
Hum, yes...
But what about my earlier problem about blackout period sync between primary and secondary?
I don't want a final host to be backuped by both backuppc at the the same time, IO would be really poor!

I'm still interesting in a scheduler simulator! :-)
May be some developer could point me, in the backuppc code?

I can tolerate poor IO on the primary backuppc, which is dedicated to backup.
Can I use a DumpPreUserCmd , or some other check, to schedule a deferred backup on a given host?

For now my script should work, except a small permission problem during tar extract on the secondary.
I'm going to perform everything as root, should kept user's id and original permission, as backuped from the final host.

Regards,
Sylvain.
-- 
Sylvain Viart.
Gmail.
------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/