Re: [Bacula-users] Large backup to tape?
> Thanks for the suggestions!
> We have a couple more questions that I hope have easy answers. So, it's
> been strongly suggested by several folks now that we back up our 200TB of
> data in smaller chunks. This is our structure:
> We have our 200TB in one directory. From there we have about 10,000
> subdirectories that each have two files in it, ranging in size between 50GB
> and 300GB (an estimate). All of those 10,000 directories adds to up about
> 200TB. It will grow to 3 or so petabytes in size over the next few years.
> Does anyone have an idea of how to break that up logically within bacula,
> such that we could just do a bunch of smaller "Full" backups of smaller
> chunks of the data? The data will never change, and will just be added to.
> in, we will be adding more subdirectories with 2 files in them to the main
> directory, but will never delete or change any of the old data.
> Is there a way to tell bacula to "back up all this, but do it in small 6TB
> or something? So we would avoid the massive 200TB single backup job +
> hundreds of (eventual) small incrementals? Or some other idea?
> Thanks again for all the feedback! Please "reply-all" to this email when
I was in a similar situation... I had a directory that was only every appended
to, it was only around 10GB but the backup was over a 512kbps link so the
initial 10GB couldn't be reliably backed up in one hit during the after-hours
window I had available. So what I did was create a fileset that included around
5% of the files (eg aa*-an*), then progressively changed that fileset to
include more and more files each backup. The important thing here is to use the
"IgnoreFileSetChanges = yes" flag in the fileset so Bacula doesn't want to do a
full backup every time the fileset changed. This is all using incremental
backups. Once I had it all backed up it just backed up the overnight changes
and everything was good.
My situation was different though in that I was doing a weekly virtual full to
consolidate the backups into one volume, which is harder to do with 2PB of
data, but if your challenge is getting the initial data backed up in less than
a single 200TB chunk then you can do it by manipulating the fileset as long as
you have IgnoreFileSetChanges = yes.
I don't think you would need accurate=yes to do the above either.
Virtualization & Cloud Management Using Capacity Planning
Cloud computing makes use of virtualization - but cloud computing
also focuses on allowing computing to be delivered as a service.
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net