Arno Lehmann schrieb:

> 27.11.2009 13:23, Ralf Gross wrote:
> > [crosspost to -users and -devel list]
> >
> > Hi,
> >
> > we are happily using bacula since a few years and already backing up
> > some dozens of TB (large video files) to tape.
> >
> > In the next 2-3 years the amount of data will be growing to 300+ TB.
> > We are looking for some very pricy solutions for the primary storage
> > at the moment (NetApp etc). But we (I) are also looking if it is
> > possible to go on with the way we store the data right now. Which is
> > just some large raid arrays and backup2tape.
>
> Good luck... while I agree that SAN/NAS appliances tend to look
> expensive, they've got their advantages when your space has to grow to
> really big sizes. Managing only one setup, when some physical disk
> arrays work together is one of these advantages.

I fully agree. But this comes with a price that is 5-10 time higher
than a setup with simple RAID arrays and a large changer. In the end
I'll present 2 or 3 concepts and others will decide how valuable the
data is.

> Also, if you're running a number of big RAID arrays, reliable vendor
> support is surely beneficial.
>
> > I've no problem with the primary storage and 10 very big raid boxes
> > (high availability is not needed).  What frightens me is to backup all
> > the data. Most files will be written once and maybe never been
> > accessed again.
>
> Yearly full backups, and then only incrementals (using accurate backup
> mode) should be a usable approach. Depending on how often you expect
> to need recovery, you may want your boss to spend some money on a
> bigger tape library :-) to make sure most data can be restored without
> manually loading tapes.

A big 500 slot lib ist already part of the idea.

> > But the data need to be online and there is a
> > requirement for backup and the ability to restore deleted files
> > (retention time can be different, going from 6 months to a couple of
> > years).
>
> 6 months retention time and actually pruning data would be easier with
> full backups more often than one year, probably.
>
> I think you should start by defining how long you want to keep your
> data, how to do full backups when those jobs will surely run longer
> than your regular backup windows (either splitting the jobs into
> smaller parts, or making sure you can run backups over a
> non-production network; measuring impact of backups on other file
> system accesses).

Some of the data will only be for a couple of months on the filer,
some for a couple of years. The filer(s) won't be that busy, and there
is a dedicated LAN for backups.

> > The snapshot feature of some commerical products is a very nice
> > feature for taking backups and it's a huge benefit that only the
> > deltas are stored.
>
> You can build on snapshot capability of SAN filers with Bacula. You'll
> still get normal file backups, but that's an advantage IMO... the most
> useful aspect of those snapshots is that you get a consistent stae of
> the file system, and don't affect production access more than necessary.
>
> > Backup2tape means that with the classic Full/Diff/Incr setup we'll
> > need many tapes. Even if the data on the primary storage won't change.
>
> Sure - a backup is reliable if you've got at least two copies of your
> files, so for 300 TB, you'll need some tapes. But tapes are probably
> cheaper than the required disk capacity for a NetApp filer :-)

Compared to a NetApp filer, tapes are definitly cheaper. Using cheaper
raid arrays it might be a bit different.