BackupPC-users

Re: [BackupPC-users] BackupPC-Visualize - visual plot of host backup durations

2013-02-06 06:05:06
Subject: Re: [BackupPC-users] BackupPC-Visualize - visual plot of host backup durations
From: Jonathan Schaeffer <jonathan.schaeffer AT univ-brest DOT fr>
To: backuppc-users AT lists.sourceforge DOT net
Date: Wed, 06 Feb 2013 12:04:11 +0100
Le 28/01/2013 18:46, Carl Cravens a écrit :
> I've written a little tool to help analyze BackupPC "scheduling" that I 
> thought others might find useful.  It is meant to generate plots that can be 
> viewed from the web, but it's just as easily used from the command line.
>
> https://github.com/ravenx99/backuppc-visualize/
>
> Over the past year+ that we've been using BackupPC at my company, we've 
> struggled with getting all our machines backed up every 24 hours.  We have 
> over 60 Linux and Windows hosts, and over half of those are virtual machines 
> all attached to a single EMC storage server, and a single full backup of all 
> the hosts is at 2 TB of data after dedupe.
>
> Because of the load backups create on our VMs, our default is to blackout 6 
> AM to 6 PM, and then allow only trivial machines to back up during the day.  
> We restrict the number of concurrent backups to 3, due to the serious load it 
> creates on a VM host when more than one of its guests is being backed up at 
> the same time.
>
> This has caused a problem with getting backups of critical machines on a 
> 24-hour cycle.  In order to see how the backups are interacting, I wrote 
> BackupPC-Visualize to create a broken-bar plot (a "timeline") indicating the 
> time during which each backup ran over the last N days.  This aggregate view 
> of the data quickly identified our problem machines.  (Our solution has been 
> to use cron to force the full backups of problem machines to run on the 
> weekend.  A kludge, and I'd like to find a more elegant solution.)
>
> bpcviz consists of a simple Perl script that gathers and massages the 
> /var/lib/backuppc/pc/<host>/backups data, and a Ploticus plot script to 
> convert the data to visual form.  I tried to use Gnuplot initially, but it 
> won't do horizontal plots, and it handles multiple-interval time data very 
> poorly (via a kludge).
>
> Let me know if you find it useful, and I'm open to considering feature 
> requests... you can open an "issue" on Github tagged as "enhancement", or 
> just drop me an email if Github is a barrier for you.
>
> (I hope the author of BackupPC doesn't object to the name... it's the most 
> obvious way to make it clear that this tool works with BackupPC.)
>

This is realy a good idea. Easy to use and easy to interpret. Thank you.

This is my small contribution :
to avoid the error when parsing a host that has never been backed up, 
issue a warn instead of die :

diff --git a/bpcviz-gatherdata b/bpcviz-gatherdata
index e796637..2d1b909 100755
--- a/bpcviz-gatherdata
+++ b/bpcviz-gatherdata
@@ -88,7 +88,8 @@ my @data = ();
  # merge all the data files to sort them by time
  for my $host (@hosts) {
      my $file = "$hostdir/$host/backups";
-    open( my $IN, '<', $file ) || die( "Cannot open $file: $!\n" );
+    unless (open $IN,'<', $file )
+      { warn( "Cannot open $file: $!\n" ); next ; }
      while ( my $line = <$IN> ) {
          chomp($line);
          my @rec=split(/\t/, $line);

cheers
-- 
IUEM - Service Informatique
rue Dumont D'Urville
Technopôle Brest-Iroise
29280 Plouzané
France
tel: +33 2 98 49 87 94

------------------------------------------------------------------------------
Free Next-Gen Firewall Hardware Offer
Buy your Sophos next-gen firewall before the end March 2013 
and get the hardware for free! Learn more.
http://p.sf.net/sfu/sophos-d2d-feb
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/