Bacula-users

[Bacula-users] An idea for a feature (not a request yet :) - weighted concurrency

2009-03-20 06:30:11
Subject: [Bacula-users] An idea for a feature (not a request yet :) - weighted concurrency
From: "James Harper" <james.harper AT bendigoit.com DOT au>
To: <bacula-users AT lists.sourceforge DOT net>
Date: Fri, 20 Mar 2009 21:24:21 +1100
I have half an idea for a feature request but it's not well defined
yet...

Basically, I have a bunch of clients to back up, some are on gigabit
network and some are stuck on 100mbit. They are being backed up to a
disk that has throughput of around 20-30mbytes/second.

I am allowing 2 jobs to run at once, which works well but I think it
could work better. I was thinking of something along the lines of
assigning a 'cost' to each client, and have the scheduler make sure that
the sum of all current clients is under some threshold, something like:

Client1 - gigabit network and fast disks - cost = 100
Client2 - gigabit network and slow disks - cost = 50
Client3 - 100mbit network - cost = 20
Client4 - 100mbit network - cost = 20
Client5 - gigabit network and fast disks - cost = 100
Client6 - gigabit network and fast disks - cost = 100

Client1,5, and 6 are pretty much capable of saturating the backup medium
on their own, Client2,3, and 4 are not. I could then set a maximum
concurrent job 'cost' of 150, and bacula would run as many jobs as it
could concurrently as long as the total cost remained <= than that
figure. So...

Client1 and 2 would start running (total cost = 150)
Client1 would finish and Client3 and 4 would start (total cost = 90)
Client2 would finish and Client5 would start (total cost = 140)
Client3 would finish
Client4 would finish
Client5 would finish and then Client6 would start

What I'm finding now is that once the two slowest jobs start running,
there is idle bandwidth on the backup medium until one of them finishes
and a faster one starts. The problem gets worse once you start thinking
about maybe backing up some data over a wan or something. Maybe I could
increase the concurrent jobs to 3 or more but I'd like to minimise the
backup window of each client if I can.

Comments?

James

------------------------------------------------------------------------------
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

<Prev in Thread] Current Thread [Next in Thread>