On Friday 27 November 2009 13:23:38 Ralf Gross wrote:
> [crosspost to -users and -devel list]
>
> Hi,
>
> we are happily using bacula since a few years and already backing up
> some dozens of TB (large video files) to tape.
>
> In the next 2-3 years the amount of data will be growing to 300+ TB.
> We are looking for some very pricy solutions for the primary storage
> at the moment (NetApp etc). But we (I) are also looking if it is
> possible to go on with the way we store the data right now. Which is
> just some large raid arrays and backup2tape.
>
> I've no problem with the primary storage and 10 very big raid boxes
> (high availability is not needed).  What frightens me is to backup all
> the data. Most files will be written once and maybe never been
> accessed again. But the data need to be online and there is a
> requirement for backup and the ability to restore deleted files
> (retention time can be different, going from 6 months to a couple of
> years).
>
> The snapshot feature of some commerical products is a very nice
> feature for taking backups and it's a huge benefit that only the
> deltas are stored.
>
> Backup2tape means that with the classic Full/Diff/Incr setup we'll
> need many tapes. Even if the data on the primary storage won't change.
>
> Backup2disk has the disadvantage that a corrupt filesystem (been
> there, seen that...) can ruin TB's of backed up data. And we will need
> file storage that is much bigger than the primary storage (keeping x
> versions of a file....).
>
> Anyone else here with the same problem? Anyone (maybe Kern or Eric)
> here that can tell if one of the upcoming new bacula features (dedup?)
> could help to solve the problem with the massive amount of tapes
> needed and the growing time windows and bandwidth for backups?

Last week Eric and I visited a customer that has the following in just 1 of 5 
datacenters:
- 10TB/day backed up
- 800+ Jobs/day
- 125TB total data backed up
- Growing rapidly
- No tape backup (a bit unusual)

It does take some big iron to run such operations, but Bacula can and is 
handling it. Obviously it takes professional configuration and tuning of the 
hardware, systems, and Bacula.

We are discussing various ways of doing deduplication, better Windows backup, 
better and faster bare metal recovery, better reporting, archival storage, 
multiple datacenters coordination, ... as part of development plans for 2010.

The above is only one example.  There are a lot of such sites out there, some 
we know about, and others that are doing their own thing with Bacula.  Many 
want significant funded development, which will provide opportunities for 
Bacula programmers (if anyone is a Windows expert programmer, please contact 
me).  

Since we (Bacula Systems) are an full Open Source company, this will also add 
a nice number of new features and capabilities for Bacula, since all code 
created for funded development goes into the project.

Good luck,

Kern

PS: tip -- a key to reducing time windows is to ensure you have the right 
bandwidth into the SD and that your catalog is properly tuned (probably 
running PostgreSQL).

>
> Any thought?
>
> Thanks, Ralf
>
> ---------------------------------------------------------------------------
>--- Let Crystal Reports handle the reporting - Free Crystal Reports 2008
> 30-Day trial. Simplify your report design, integration and deployment - and
> focus on what you do best, core application coding. Discover what's new
> with Crystal Reports now.  http://p.sf.net/sfu/bobj-july
> _______________________________________________
> Bacula-devel mailing list
> Bacula-devel AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-devel



------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users