Bacula-users

Re: [Bacula-users] Pool per client

2011-04-13 13:58:45
Subject: Re: [Bacula-users] Pool per client
From: Pablo Marques <pmarques AT miamilinux DOT net>
To: "Kurzawa, Greg" <GKurzawa AT pamida DOT com>
Date: Wed, 13 Apr 2011 13:55:28 -0400
Thanks Greg,

The key in my application is if I can modify bacula-sd.conf dynamically 
without a bacula-sd restart every time I add or remove a client.

bacula-sd restart would kill running jobs.

Pablo

Kurzawa, Greg wrote on Wednesday, April 13, 2011
> Pablo,
> 
> 
> 
> If you’re backing up to disk, you can have as many Jobs running
> simultaneously as you want/need.  The trick here is to define not only a
> unique Pool resource for each Client, but unique Storage, Device and Media
> Type resources as well.  Each Client will have its own resources, so there
> will be no contention.  Following are the relevant pieces of my
> configuration, snipped for readability.
> 
> 
> 
> I admit that when I was setting this up it seemed a bit bulky, but it
> works.  I back up many slow Clients to disk; their jobs all begin and run
> simultaneously.
> 


 
> 
> Of course, if you can’t back up to disk first, this is useless to you.
> 
> 
> 
> #### CLIENTS ####
> 
> 
> 
> Client {
> 
>   Name = DORCAS
> 
>   Address = DORCAS.URTH.COM
> 
> }
> 
> 
> 
> Client {
> 
>   Name = SEVERIAN
> 
>   Address = SEVERIAN.URTH.COM
> 
> }
> 
> 
> 
> #### JOBS ####
> 
> 
> 
> Job {
> 
>   name = "DAILY:severian"
> 
>   client = "SEVERIAN"
> 
>   pool = "SEVERIAN"
> 
> }
> 
> 
> 
> Job {
> 
>   name = "DAILY:dorcas"
> 
>   client = "DORCAS"
> 
>   pool = "DORCAS"
> 
> }
> 
> 
> 
> #### POOLS ####
> 
> 
> 
> Pool {
> 
>   Name = "SEVERIAN"
> 
>   Pool Type = Backup
> 
>   Storage = "SAN:severian"
> 
>   next pool = "LTO4"
> 
>   LabelFormat = "severian-"
> 
> }
> 
> 
> 
> Pool {
> 
>   Name = "DORCAS"
> 
>   Pool Type = Backup
> 
>   Storage = "SAN:dorcas"
> 
>   next pool = "LTO4"
> 
>   LabelFormat = "dorcas-"
> 
> }
> 
> 
> 
> #### STORAGE ####
> 
> 
> 
> Storage {
> 
>   Name = "SAN:severian"
> 
>   Device = "SAN:severian"
> 
>   Media Type = "FILE:severian"
> 
> }
> 
> 
> 
> Storage {
> 
>   Name = "SAN:dorcas"
> 
>   Device = "SAN:dorcas"
> 
>   Media Type = "FILE:dorcas"
> 
> }
> 
> 
> 
> And in bacula-sd.conf:
> 
> 
> 
> Device {
> 
>   Name = "SAN:severian"
> 
>   Media Type = "FILE:severian"
> 
>   Archive Device = "/dp-SAN/severian"
> 
> }
> 
> 
> 
> Device {
> 
>   Name = "SAN:dorcas"
> 
>   Media Type = "FILE:dorcas"
> 
>   Archive Device = "/dp-SAN/dorcas"
> 
> }
> 
> 
> 
> Greg
> 
> 
> 
> 
> 
> From: Pablo Marques [mailto:pmarques AT miamilinux DOT net]
> Sent: Wednesday, April 13, 2011 11:01 AM
> To: Kurzawa, Greg
> Cc: bacula-users AT lists.sourceforge DOT net
> Subject: Re: [Bacula-users] Pool per client
> 
> 
> 
> Thanks Greg,
> 
> But I would still have the problem that I need a device tied up backing up
> each client. The problem I am facing is that I need to backup lots of slow
> clients, and I need to come up with something so I can back them up all at
> the _same_ time on one or maybe a few devices, and still have a Pool per
> client.
> 
> Pablo
> 
> ________________________________
> 
> From: "Greg Kurzawa" <GKurzawa AT pamida DOT com>
> To: "Pablo Marques" <pmarques AT miamilinux DOT net>
> Cc: bacula-users AT lists.sourceforge DOT net
> Sent: Wednesday, April 13, 2011 9:29:31 AM
> Subject: RE: [Bacula-users] Pool per client
> 
> 
> 
> 
> Hi Pablo,
> 
> 
> 
> If you have enough disk space handy, you could send each Client’s data to
> its own disk Pool with its own Next Pool specification.  Each Client’s
> data would be in its own Pool on disk, then move to its own Pool on tape. 
> This is exactly what I’ve done at my site, except the disk Pools all point
> to the same tape Pool.
> 
> 
> 
> Greg
> 
> 
> 
> From: Pablo Marques [mailto:pmarques AT miamilinux DOT net]
> Sent: Wednesday, April 13, 2011 7:28 AM
> To: Randy Katz
> Cc: bacula-users AT lists.sourceforge DOT net
> Subject: Re: [Bacula-users] Pool per client
> 
> 
> 
> Thanks Randy,
> 
> Unfortunately Maximum Concurrent Jobs won't cut it because I need a
> different tape per client.  Still I set it to 1000. When spooling is
> enabled, bacula still wants a tape from the client pool on the drive
> before the spooling starts.
> 
> I could create a virtual library with MHVTL and use several drives or use a
> disk-changer emulator. But I am not sure how scalable these solutions are.
> 
> Pablo
> 
> ________________________________
> 
> From: "Randy Katz" <rkatz AT simplicityhosting DOT com>
> To: bacula-users AT lists.sourceforge DOT net
> Sent: Wednesday, April 13, 2011 6:08:54 AM
> Subject: Re: [Bacula-users] Pool per client
> 
> On 4/12/2011 4:04 PM, Pablo Marques wrote:
> 
> I have a setup to backup lots of clients over slow links.
> I want to have each client (or group of clients) backed up to dedicated
> client pools, so client1 will go to pool client1, and so on. That way I
> have better control of the space used, if a client goes away I can simply
> delete the tapes (or files) an get the space back immediately. Also it
> gives me better control on the retention on a per client basis.
> 
> The problem is that when I try to backup multiple clients at the same time,
> the storage process has to wait for each job to finish before it can move
> to the next because it needs to change the tape (different client -->
> different pool). Some clients may take many hours to finish, forcing
> everybody else to wait.
> 
> I enabled spooling, but it seems like Bacula requires to mount a tape from
> the client pool on a drive before the client spooling can begin. Can this
> be avoided?
> 
> A possible solution would be to do all backups on a special pool and after
> they are done migrate later each client job to each client pool. But I
> cannot find a way to modify  the "Next Pool" dynamically. It is a fixed
> setting on the Pool definition.
> 
> Does anybody have suggestions on how to accomplish this?
> 
> Look into Maximum Concurrent Jobs in your Storage definition.
> 
> Regards,
> Randy
> 
> ---------------------------------------------------------------------------
> --- Forrester Wave Report - Recovery time is now measured in hours and
> minutes not days. Key insights are discussed in the 2010 Forrester Wave
> Report as part of an in-depth evaluation of disaster recovery service
> providers. Forrester found the best-in-class provider in terms of services
> and vision. Read this report now!  http://p.sf.net/sfu/ibm-webcastpromo
> _______________________________________________
> Bacula-users mailing list
> Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users

------------------------------------------------------------------------------
Forrester Wave Report - Recovery time is now measured in hours and minutes
not days. Key insights are discussed in the 2010 Forrester Wave Report as
part of an in-depth evaluation of disaster recovery service providers.
Forrester found the best-in-class provider in terms of services and vision.
Read this report now!  http://p.sf.net/sfu/ibm-webcastpromo
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users