Bacula-users

Re: [Bacula-users] Job is waiting on max Storage jobs although I set Max concurrent job to 10

2011-01-13 15:14:13
Subject: Re: [Bacula-users] Job is waiting on max Storage jobs although I set Max concurrent job to 10
From: Silver Salonen <silver AT ultrasoft DOT ee>
To: bacula-users AT lists.sourceforge DOT net
Date: Thu, 13 Jan 2011 22:10:58 +0200
On Thursday 13 January 2011 17:26:55 Hugo Letemplier wrote:
> 2011/1/12 Silver Salonen <silver AT ultrasoft DOT ee>:
> > On Wednesday 12 January 2011 13:56:44 Hugo Letemplier wrote:
> >> 2011/1/8 Silver Salonen <silver AT ultrasoft DOT ee>:
> >> > On Saturday 08 January 2011 14:07:01 Hugo Letemplier wrote:
> >> >> Le 6 janv. 2011 à 21:17, Silver Salonen a écrit :
> >> >>
> >> >> > On Thursday 06 January 2011 22:14:16 John Drescher wrote:
> >> >> >>> Only one job can run on the same device. If you want another job to 
> >> >> >>> run into the save folder, you have to clone the device resource 
> >> >> >>> under some other name. Eg:
> >> >> >>>
> >> >> >>
> >> >> >> That is wrong. Multiple jobs can write to the same tape or disk 
> >> >> >> volume
> >> >> >> at the same time. Most likely your problem is that a single bacula
> >> >> >> storage device can only load 1 volume and thus 1 pool at a time even
> >> >> >> for disk volumes.
> >> >> >>
> >> >> >> John
> >> >> >
> >> >> > Yeah, sry, I forgot that. I wonder who and why writes multiple jobs 
> >> >> > into the same volume with file-type volumes :)
> >> >> >
> >> >> > --
> >> >> > Silver
> >> >>
> >> >> So I will need to create many pools ?
> >> >> But of I use one pool for many Jobs so I will need to create many 
> >> >> devices ?
> >> >>
> >> >> Hugo
> >> >
> >> > Many pools you need mostly because you want your backups of different 
> >> > levels to be labeled differently and have different retention time for 
> >> > them.
> >> >
> >> > For file-type volumes it makes sense to use one job per volume and thus 
> >> > you'll need to create as many devices as many concurrent jobs you want 
> >> > to use.
> >> >
> >> > PS. In mailing listis, please post at the bottom of the thread.
> >> >
> >> > --
> >> > Silver
> >> >
> >>
> >> Do I need to use a virtual autochanger ? in fact I don't care about
> >> what device I used per pool.
> >> eg : if I have many file devices I order  to access many file volumes
> >> in one sd directory. I would like that bacula select automaticatly the
> >> first available device in order to access the file volume.
> >>
> >> Here are my File devices in sd conf
> >>
> >> Device {
> >>   Name = "DataTest.0"
> >>   Media Type = File
> >>   Archive Device = /BaculaData/Test/
> >>   LabelMedia = yes;                   # lets Bacula label unlabeled media
> >>   Random Access = Yes;
> >>   AutomaticMount = yes;               # when device opened, read it
> >>   RemovableMedia = no;
> >>   AlwaysOpen = Yes;
> >>   Maximum Volume Size = 4G
> >> }
> >>
> >> Device {
> >>   Name = "DataTest.1"
> >>   Media Type = File
> >>   Archive Device = /BaculaData/Test/
> >>   LabelMedia = yes;                   # lets Bacula label unlabeled media
> >>   Random Access = Yes;
> >>   AutomaticMount = yes;               # when device opened, read it
> >>   RemovableMedia = no;
> >>   AlwaysOpen = Yes;
> >>   Maximum Volume Size = 4G
> >> }
> >>
> >> imagine 10 Devices named "DataTest.X"
> >>
> >> Here are my pools in Dir config, one named "Sauvegarde" is to make
> >> regular backup on few File servers. The other is to export on a LTO
> >> device as long as possible.
> >>
> >> Pool {
> >>       Name = Test
> >>       Storage = DataTest   => My problem is here
> >>       Pool Type = Backup
> >>       Recycle = yes                                                        
> >>    # Bacula can automatically recycle Volumes
> >>       AutoPrune = yes                                                      
> >>    # Prune expired volumes
> >>       Volume Retention = 7 days                                # Short 
> >> time to make test
> >>       Label Format = "Bacula-Test-"
> >>       Maximum Volume Bytes = 4G                           # Limit Volume 
> >> size to
> >> something reasonable
> >>       Maximum Volumes = 256                                           # 
> >> Limit number of Volumes in Pool
> >>       Next Pool = ArchivageLTO
> >> }
> >>
> >> Pool {                                                                     
> >>            #Pool pour l'archivage sur bande LTO
> >>       Name = ArchivageLTO
> >>       Storage = Tape   => My problem is here
> >>       Pool Type = Backup
> >>       Label Format = "Bacula-LTO-"
> >>       Recycle = no                                                         
> >>    # Bacula can automatically recycle Volumes
> >>       AutoPrune = no                                                       
> >>    # Prune expired volumes
> >>       Maximum Volumes = 0                                                  
> >>    # unlimited number of Volumes in Pool
> >> }
> >>
> >> => My problem is here = I can only specify 1 device .
> >>
> >> You said one pool per job but I will have many job with the same
> >> retention time and space limit.
> >> I want to backup my File servers as long as it is possible with the
> >> only limit of the space on the Disk.
> >>
> >> I was going to make Pools in function of the kind of Data to use and
> >> the "Value" of the data stored on it.
> >> For eg : Financial and administrative Data of my company in a
> >> different pool than the Webserver, the File Server or than the Mail
> >> Server.
> >>
> >> My LTO pool is only the destination of a copy job, in this case this
> >> is an archive and I need to maybe 20 years later so I configure it
> >> with an unlimited retention time.
> >>
> >> Do I really need to make one pool per job ?
> >
> > As I have understood it, the problem will occur when you are going to 
> > rotate your backups (eventually you WILL do it, no?). I have separated 
> > different data to separate jobs (eg. configuration, /var, /home etc.) and 
> > they all have different retention period and/or different number of backups 
> > (eg. volumes of full backups). So I want to be able to rotate the backups 
> > one-by-one, not all-at-once, and this is also the reason I'm using one 
> > volume per job.
> >
> > Concerning devices - yes, you could use vchanger instead of creating many 
> > devices manually.
> >
> > --
> > Silver
> >
> 
> My idea is now to have one pool for each part of my Information system
> ex File Servers, Database Server, Infrastructure, Mails, Laptops,
> Development, Production…
> 
> For each pool I create one Device. Why would this create problem in
> backup rotation ?

A pool consists of volumes and if you use one pool for multiple jobs, you end 
up using the same volumes for multiple jobs too. If you want your jobs to be 
able to write concurrently into the same pool, you will have to enable more 
than 1 job per volume. And if you have more than 1 job in a volume, you'll have 
a problem rotating volumes of individual jobs (because you won't have one). If 
that logic is somehow incorrect, please correct me :)

But if you use one pool for one type of data, you'll have to put every level of 
the job into the same pool aka volumes. Rotating volumes of incremental, 
differential and full levels "randomly" you'll have a rotation chaos :)

So to my mind you still need one pool for every level of your every job. And 
one device for a job (not pool).

-- 
Silver

------------------------------------------------------------------------------
Protect Your Site and Customers from Malware Attacks
Learn about various malware tactics and how to avoid them. Understand 
malware threats, the impact they can have on your business, and how you 
can protect your company and customers by using code signing.
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users