On 03/05/10 10:40, Matija Nalis wrote:
> To make a clear picture, imagine the following groups of hosts:
> 
> Group A = hostA1, hostA2, hostA3, ... (those are on one xen physical machine)
> Group B = hostB1, hostB2, hostB3, ... (those are on other xen phys. machine)
> Group C = all other hosts. (those are all other [non-virtual] hosts).
> 
> What we want to do is to have absolute maximum concurrency possible (of up
> to 50 hosts backing up at once), with only restriction that at any one time,
> *at most* 1 host from group A will be running, and *at most* 1 host from group
> B will be running. There is no limit on number of group C hosts that may be
> running at the same time (other than global concurrency limit of 50 hosts,
> of course)
> 
> So, for example, at any moment, having 48 backups running from group C, 1
> from group A and 1 from group B is OK.
> So is 49 from group C, and 1 from group A. 
> 
> But 48 from group C, and 2 from group A is NOT ok.

[...]

> which would give us some head start of 50 parallel "C" hosts, and serialize
> A and B groups (as host with prio21 will wait for host with prio20 to finish
> first, and those for prio22 will wait for prio21 and prio20 etc)
> 
> But as soon as groups A and B start running, the "C" group would slowly
> starve - the initial 50 C-hosts will end one by one, but new ones won't be
> starting until both group A and B finish (which could take quite some time
> in our situation -- so we would waste a good part of the backup window
> running just one "A" host and one "B" host at the same time).
> 
> Anybody has a better idea ? Thanks!

Why don't you try this (I'm assuming you're backing up to disk, as you
didn't specify):

Set up three Storage devices on your Storage daemon, and three Pools,
each tied to one of the Storage devices.  So you have Pool A on Storage
device A, Pool B on Storage device B, Pool C on Storage device C.  (They
can all point to the same physical disk pool.  That's OK.)  Assign the
clients from group A to use pool A, group B to pool B, group C to pool
C.  Then set maximum concurrency to 50 on Storage C, and to 1 on Storage
A and B.  To achieve the same effect, you could use the same basic setup
and limit Pools A and B to one Job per volume.  Either way, any number
of Group C backups (up to 50) will be able to run at once, but only one
each from groups A and B, without having to play Priority games.


-- 
  Phil Stracchino, CDK#2     DoD#299792458     ICBM: 43.5607, -71.355
  alaric AT caerllewys DOT net   alaric AT metrocast DOT net   phil AT 
co.ordinate DOT org
         Renaissance Man, Unix ronin, Perl hacker, Free Stater
                 It's not the years, it's the mileage.

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users