Bacula-users

Re: [Bacula-users] Benefits of Hardware compression versus Software compression for highly compressable data

2013-05-14 20:11:09
Subject: Re: [Bacula-users] Benefits of Hardware compression versus Software compression for highly compressable data
From: James Harper <james.harper AT bendigoit.com DOT au>
To: Raimund Sacherer <raimund.sacherer AT logitravel DOT com>, bacula-users <bacula-users AT lists.sourceforge DOT net>
Date: Wed, 15 May 2013 00:06:17 +0000
> Hello,
> 
> we do daily fullbackups for our database backups and currently I use
> software compression. The reason behind this is that I do disk2disk2tape and
> I want 2 weeks of backups readily on disk for fast restore.
> 
> Four our databases we do the following (we have around 500 gig of DB
> Backup data each day):
> 

You mean hardware compression performed by the tape drive itself right?

For backup throughput you have to consider:
. How fast can data be read off the disk (and compressed) on the fd
. How fast can data be transmitted to the sd
. How fast can the sd write out the data to the storage medium

If your fd->sd link is slow (eg 1mbit broadband or something) then software 
compression is a clear win as it happens at the fd and increases the effective 
backup throughput.

If you fd->sd link is fast then compression is likely going to slow you down. 
With compression on I max out at around 2mbytes/second, vs 20-50mbytes/second 
when compression is off. I'm mainly backing up to USB2 attached storage which 
peaks around 20mbytes/second write speed anyway, and I need the storage space, 
so I use compression.

Hardware compression happens on the tape drive itself so obviously less load on 
the fd's, but higher bandwidth utilisation. If your data is compressed on disk 
on your sd (because your fd's compressed it) and your sd has high speed disks 
that can feed data to the tape drive as fast as it can take it then your backup 
is going to be faster.

Lots of variables to consider! If it was my setup I'd be doing the compression 
at the fd's and turning off hardware compression.

I think there is a new compression algorithm available in newer versions of 
Bacula but I'm not sure it made it into the windows version yet. If this is 
significantly faster then it would make fd compression a better option again.

I'd like to see sd-side compression - then you could offload to a GPU or 
dedicated encryption/compression hardware and really fly!

James

------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users