> Thanks for the suggestions!
> 
> We have a couple more questions that I hope have easy answers.  So, it's
> been strongly suggested by several folks now that we back up our 200TB of
> data in smaller chunks.  This is our structure:
> 
> We have our 200TB in one directory.  From there we have about 10,000
> subdirectories that each have two files in it, ranging in size between 50GB
> and 300GB (an estimate).  All of those 10,000 directories adds to up about
> 200TB.  It will grow to 3 or so petabytes in size over the next few years.
> 
> Does anyone have an idea of how to break that up logically within bacula,
> such that we could just do a bunch of smaller "Full" backups of smaller
> chunks of the data?  The data will never change, and will just be added to.  
> As
> in, we will be adding more subdirectories with 2 files in them to the main
> directory, but will never delete or change any of the old data.
> 
> Is there a way to tell bacula to "back up all this, but do it in small 6TB 
> chunks"
> or something?  So we would avoid the massive 200TB single backup job +
> hundreds of (eventual) small incrementals?  Or some other idea?
> 
> Thanks again for all the feedback!  Please "reply-all" to this email when
> replying.
> 

I was in a similar situation... I had a directory that was only every appended 
to, it was only around 10GB but the backup was over a 512kbps link so the 
initial 10GB couldn't be reliably backed up in one hit during the after-hours 
window I had available. So what I did was create a fileset that included around 
5% of the files (eg aa*-an*), then progressively changed that fileset to 
include more and more files each backup. The important thing here is to use the 
"IgnoreFileSetChanges = yes" flag in the fileset so Bacula doesn't want to do a 
full backup every time the fileset changed. This is all using incremental 
backups. Once I had it all backed up it just backed up the overnight changes 
and everything was good.

My situation was different though in that I was doing a weekly virtual full to 
consolidate the backups into one volume, which is harder to do with 2PB of 
data, but if your challenge is getting the initial data backed up in less than 
a single 200TB chunk then you can do it by manipulating the fileset as long as 
you have IgnoreFileSetChanges = yes.

I don't think you would need accurate=yes to do the above either.

James


------------------------------------------------------------------------------
Virtualization & Cloud Management Using Capacity Planning
Cloud computing makes use of virtualization - but cloud computing 
also focuses on allowing computing to be delivered as a service.
http://www.accelacomm.com/jaw/sfnl/114/51521223/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users