Bacula-users

Re: [Bacula-users] Benefits of Hardware compression versus Software compression for highly compressable data

2013-05-14 11:13:15
Subject: Re: [Bacula-users] Benefits of Hardware compression versus Software compression for highly compressable data
From: Radosław Korzeniewski <radoslaw AT korzeniewski DOT net>
To: Raimund Sacherer <raimund.sacherer AT logitravel DOT com>
Date: Tue, 14 May 2013 17:09:20 +0200
Hello,

2013/5/14 Raimund Sacherer <raimund.sacherer AT logitravel DOT com>
Hello,

we do daily fullbackups for our database backups and currently I use software compression. The reason behind this is that I do disk2disk2tape and I want 2 weeks of backups readily on disk for fast restore.

Four our databases we do the following (we have around 500 gig of DB Backup data each day):

The database servers backup their databases locally.
-> Each time a database has finished backup it's backup to the local drive, we copy it over the backup server
-> After all databases of a server have been backed up locally and copied to the backup server we launch a bacula job
-> Older backups (2-3 days, depends) are deleted from the local backup

The bacula job then:
-> figures out all the database files backed up in the last 20 hours
-> backs them up to disk with software compression on (I get around 82% compression ratio)
-> deletes source backups which are older than 2 days from the backup server

In the morning, all the backup jobs get copied to tape for. We rotate daily tapes 2 weeks and Weekly tapes 10 weeks.


Now, for databases that leaves me:

2-3 days worth of uncompressed backups on the database server itself (easy and fastest restore)
2 days worth of uncompressed backups on the backup server (fast restore in case DB server died horribly) 
14 days worth of compressed backups on the backup servers disk (slow restore, but conviniently on disk)

I believe Bacula support client only software compression, so could you describe how do you achieve a backups compression on backup server (Bacula Storage Daemon)? Is it some kind of filesystem level compression or do you do something else?
 
10 weeks worth of weekly compressed backups on the tapes (slowest restore, but at least you can have data from way back)


It does suit our needs nicely but creating the compressed software backups costs a lot of time, it would go way faster if I just did hardware compression on the tapes, but then I would have the problem with local diskspace. keeping 14 days of uncompressed data amounts to about 7 Terrabyte versus the roughly 140 Gig's I get from the (avg. 82%) compression.

How do you handle big database backups? Is it worth storing uncompressed?

I store database backups only on backup server with D2D2T functionality. Uncompressed on backup server disks and compressed on tapes (hardware compression).
 
Is the hardware compression *actually* better?

Hardware compression is not better then software one, but a way faster... :)
 
Do I waste space on tape?

If you move already compressed data to the tape then it is not a waste of tape space. You can't get any better.
 
best regards
--
Radosław Korzeniewski
radoslaw AT korzeniewski DOT net
------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users