Thanks a lot Dan for your help and your
helpful website (I went there before asking for information here:
dan.langille.org/2010/02/08/bacula-retention-periods/).
Le 20/10/2012 12:35, Dan Langille a écrit :
On Oct 18, 2012, at 12:54 PM, Florent Krieg wrote:
Hi Dan,
Thanks a lot for your help, it has gone much clearer to
me.
I still have a few questions left:
Le 17/10/2012 21:55, Dan Langille a écrit :
On Oct 17,
2012, at 11:30 AM, Florent Krieg wrote:
Hi
there!
We are currently using bacula for a while (3/4 years
maybe) to backup many
servers (VMs as weel as physical machines) on a
storage server (volumes are
not tapes, but basically only labelled files of 1GB
size).
We don't have a complicated architecture, even if we
backup every night
around 50 servers. The only thing we need is to be
able to restore one or
more files, but having different version of the same
file is not important
to us.
So here is what we used to have (until we realized we
were wrong):
- 2 full backups a month (1st and 3rd weeks of the
month, on Saturday night)
- incremental backups otherwise
This means for a month, for instance:
FIIIIII-IIIIIII-FIIIIII-IIIIII
Everything seemed to be OK until I tried to restore
something yesterday.
The problem I found is that when we initially setuped
bacula, storage was a
huge constraint (we hadn't any storage server
actually) and every retention
parameters were set to 7 days:
- Client: file/job/volume retention = 7 days
- Pool (the same pool is used for full and incr
backups, which is ok in our
case, am I right?): volume retention = 7 days
Well, it's OK, but the problem is, any backup more that
7 days will not be listed
in the Catalog. It may well exist on disk, but without
the Catalog entries, there
is no *EASY* way to restore that
backup. And ease-of-restore is exactly what
the Catalog is there for.
Ok so
it's quite obvious we had a huge problem there. To my
mind, we should
have no problem to restore anything the week following
the full backup but
there were still two blacks weeks in the month where
we couldn't restore in
a proper way (we could restore as best effort
though... What I actually
did).
Thus to rethink our bacula configuration I've tried
since to read bacula
(5.0.0) manual and forums on the Internet, without
being able to clearly
understand what is the purpose of each retention time.
Logically I would set
the retentions to 14 days everywhere, and this should
solves the problem,
but I am not sure of that and as I could experiment,
I'd prefer understand
what I'm doing.
Retention is all about the Catalog, not necessarily the
backup.
Retention is: how long do you want to record this backup
in the Catalog.
I recommend keeping all retentions the same: Volume,
Job, File.
Ok.
Could
somebody point me to a manual section that would
explain me (without
going into the deep details of Bacula) how to set
either:
- client: file/job/volume retention parameters?
Volume retention is not a Client attribute. Job and
File are.
Yes sorry, this is how we defined them in the config
files.
'show job=SERVER_job' in bconsole drove me in the wrong
me, showing me stuff as:
--> Client: name=SUPER-VIL-1_fd
address=super-vil-1.mgt.sewan.fr FDport=9102 MaxJobs=1
JobRetention=7 days FileRetention=7 days
AutoPrune=1
--> Pool: name=SUPER-VIL-1_pool PoolType=Backup
use_cat=1 use_once=0 cat_files=1
max_vols=0 auto_prune=1 VolRetention=7 days
VolUse=0 secs recycle=1 LabelFormat=SUPER-VIL-1-
CleaningPrefix= *None* LabelType=0
RecyleOldest=0 PurgeOldest=0 ActionOnPurge=0
MaxVolJobs=10 MaxVolFiles=0 MaxVolBytes=1073741824
MigTime=0 secs MigHiBytes=0 MigLoBytes=0
JobRetention=0 secs FileRetention=4447866 years 9
months 27 days 6 hours 46 mins
Where do these values come from?
My guess: default values. Those are the entries stored in
the database tables.
I would search for all mentions of retention here
http://www.bacula.org/5.2.x-manuals/en/main/main/Configuring_Director.html#SECTION0022130000000000000000
NOTE this is the same file as the next URL, just a
different section
- pool:
which retention parameters are available and if they
are redundant
or not with the others above?
Volume retention is specified in the Pool resource. I
would search for all mentions
of retention here:
http://www.bacula.org/5.2.x-manuals/en/main/main/Configuring_Director.html#SECTION0022150000000000000000
Thanks for the link, I have read both sections carefully
and I think I got the point.
Also, I
know the scenario is really simple compared to most of
yours, but if
somebody already achieved something similar (2 or 1
full a month and then
incrementals), I'd be very greatful if he'd explain to
me how to do it.
Decide how far back you want to be able to restore a
file, then go from there.
Two months? Two years? Decide that, then set your
retention values. You'll need to update your
resources, then update your Pools based on that, using
the update command in bconsole. But that you can ask
about
later.
Read all that, then get back to the list with any
questions.
Let me know if I am wrong:
Let's consider that when a Full backup is done, I don't
mind previous backups.
A full backup is a reliable reference to me (basically
backups are just used in case of emergency in our
environment, so if a full backup of a working server is
available, that's fine).
Our schedule policy is:
Schedule {
Name = "NightlySave"
Run = Level=Full 1st,3rd sat at 06:05
Run = Level=Incremental mon-fri,sun at 04:05
Run = Level=Incremental 2nd,4th,5th sat at 04:05
}
Thus, retention times of 'time inbetween two full backups'
should definitely fit our needs, right? Say 7+7+1(error
margin)=15d ..?
I'd go with keeping more than one just full backup. I'd be
keeping 3 or 4 if you can. Things will go wrong. Plus, if a
file is mangled in Mar, and you don't find out until Nov, you
have no backups.
Two questions that I
have now are:
1. We have all AutoPrune/AutoPurge/Recycle/... parameters
set to Yes. In that case, the retention times are directly
involved in how long we keep the backup Volumes, right?
Yes.
Moreover, we don't
have any max volume files for each pool therefore if a new
volume is needed for a backup, a new one will be created
right?
Yes
2. With 7 days retention times, if I list job a server:
*list job=HSS-VIL-1_job
+--------+---------------+---------------------+------+-------+-----------+-----------------+-----------+
| JobId | Name | StartTime | Type |
Level | JobFiles | JobBytes | JobStatus |
+--------+---------------+---------------------+------+-------+-----------+-----------------+-----------+
| 40,840 | HSS-VIL-1_job | 2012-10-06 22:03:57 | B |
F | 1,681,621 | 156,228,704,503 | T |
| 40,893 | HSS-VIL-1_job | 2012-10-07 10:58:43 | B |
I | 3,066 | 325,259,836 | T |
| 40,946 | HSS-VIL-1_job | 2012-10-08 04:12:06 | B |
I | 2,150 | 119,735,689 | T |
| 40,999 | HSS-VIL-1_job | 2012-10-09 04:14:28 | B |
I | 8,345 | 466,134,754 | T |
| 41,052 | HSS-VIL-1_job | 2012-10-10 04:15:33 | B |
I | 11,125 | 946,012,469 | T |
| 41,110 | HSS-VIL-1_job | 2012-10-11 04:15:22 | B |
I | 8,898 | 604,906,727 | T |
| 41,163 | HSS-VIL-1_job | 2012-10-12 04:15:27 | B |
I | 10,610 | 613,376,814 | T |
| 41,216 | HSS-VIL-1_job | 2012-10-13 04:15:58 | B |
I | 7,604 | 530,114,112 | T |
| 41,269 | HSS-VIL-1_job | 2012-10-14 04:16:40 | B |
I | 4,086 | 554,999,485 | T |
| 41,322 | HSS-VIL-1_job | 2012-10-15 04:15:49 | B |
I | 3,275 | 198,191,930 | T |
| 41,375 | HSS-VIL-1_job | 2012-10-16 04:14:26 | B |
I | 11,713 | 601,523,941 | T |
| 41,429 | HSS-VIL-1_job | 2012-10-17 04:15:53 | B |
I | 18,210 | 838,659,657 | T |
| 41,483 | HSS-VIL-1_job | 2012-10-18 04:15:46 | B |
I | 11,810 | 575,001,874 | T |
+--------+---------------+---------------------+------+-------+-----------+-----------------+-----------+
Why do I have information about the last 12 days?
Isn't the purpose of JobRetention to clean it?
I don't know. More investigation would be required.
It seems to work on another host though:
*list job=SUPER-VIL-1_job
+--------+-----------------+---------------------+------+-------+----------+-------------+-----------+
| JobId | Name | StartTime | Type |
Level | JobFiles | JobBytes | JobStatus |
+--------+-----------------+---------------------+------+-------+----------+-------------+-----------+
| 41,179 | SUPER-VIL-1_job | 2012-10-12 04:36:44 | B |
I | 40 | 4,575,403 | T |
| 41,232 | SUPER-VIL-1_job | 2012-10-13 04:35:19 | B |
I | 67 | 4,709,002 | T |
| 41,285 | SUPER-VIL-1_job | 2012-10-14 04:27:37 | B |
I | 53 | 4,424,132 | T |
| 41,338 | SUPER-VIL-1_job | 2012-10-15 04:30:22 | B |
I | 49 | 4,526,834 | T |
| 41,391 | SUPER-VIL-1_job | 2012-10-16 04:42:28 | B |
I | 105 | 45,807,762 | T |
| 41,445 | SUPER-VIL-1_job | 2012-10-17 04:39:48 | B |
F | 38,183 | 995,992,505 | T |
| 41,501 | SUPER-VIL-1_job | 2012-10-18 04:47:52 | B |
I | 27 | 4,474,961 | T |
+--------+-----------------+---------------------+------+-------+----------+-------------+-----------+
A full backup was done two days ago because the full
backup of the first week was pruned, I reckon.
Thanks again in advance!
Florent
Thanks
in advance everybody and sorry if I feel like I am
searching
information at the wrong place.
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
|
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct _______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
|