Hi,
On Thu, 29 Jan 2015, Rao, Uthra R. (GSFC-672.0)[ADNET SYSTEMS INC]
wrote:
>
> I run bacula 5.2.12 on a RHEL server which is attached to a Tape Library. I
> have two LTO5 tape
> drives. Since the data on one of my server has grown big the back-up takes
> 10-12 days to
> complete. I would like to split this job in to two jobs. Has anybody done
> this kind of a set-up?
> I need some guidance on how to go about it.
I did something like this with our backups, dividing them up in order to
decrease the amount of time necessary for backups. At the time I conceived
this system, we had about 56TB of data. For historic reasons our backups
were made from NFS exports from a single central NAS fileserver, because
it was proprietary, I couldn't directly access the fileserver any other
way. (This has changed but I haven't made substantial changes to the
procedure yet.)
I have two tape libraries for this each with two drives, and both support
partitioning. I partitioned the two libraries so that from the computer's
perspective I have 4 libraries.
Our filesystems are composed primarily of user directories and working
group directories. I calculated the size of each of these directories and
then used a basic bin-packing algorithm to subdivide them into four groups
each being assigned to a parition. A preperation script is used to mount
each directory as a separate NFS mount in its respective "group"
directory. The groups were sized such that the ratio of the data to the
capacity of a given library's partition were more or less the same for the
four groups.
I created four pools, four jobs, for storage devices, etc., all to
match up with the 4 library paritions and four groups, essentially
creating 4 complete backup jobs.
When it's time to take new base backups, I repeat the calculation,
bin-packing part and run the update the mount points to redistribute the
data.
It did cut down the time required to do base backups substantially.
I was also able to remove a set of files that while only occupied about a
terabyte, contained tens of millions of tiny files that massively slowed
down the backup process whenever it hit that spot. Perhaps you have
similar conditions on your filesystems?
Thanks,
Rich.
--
Rich Fox
Systems Administrator
JBPC - Marine Biological Laboratory
http://www.mbl.edu/jbpc
508-289-7669 - mbl-at-richfox.org
------------------------------------------------------------------------------
Dive into the World of Parallel Programming. The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
|
