Bacula-users
[Top] [All Lists]
<prev [Date] next>
[Advanced]
 <prev [Thread] next>

Re: [Bacula-users] Files pruned from catalog or never inserted - For one or more of the JobIds selected, no files were found, so file selection is not possible.

2016-06-27 17:02:16
Subject: Re: [Bacula-users] Files pruned from catalog or never inserted - For one or more of the JobIds selected, no files were found, so file selection is not possible.
From: Ana Emília M. Arruda <emiliaarruda AT gmail DOT com>
To: Marcio Vogel Merlone dos Santos <marcio.merlone AT a1.ind DOT br>
Date: Mon, 27 Jun 2016 23:01:14 +0200
Hello Marcio,

Which was the version immediately previous to the upgrade to 7.05? In some cases, there is a need of database upgrade.

Fortunately, there are ways to restore individual files by the use of bls and bextract utilities.

With bls you can have a list of all the files backed up in one volume, the you can use bextract with "-i" option and inform here a list of files to be restored.

Also, you can use bscan to recover the jobs and media information from your volumes into your catalog again.

Best regards,
Ana




On Thu, Jun 16, 2016 at 8:00 PM, Marcio Vogel Merlone dos Santos <marcio.merlone AT a1.ind DOT br> wrote:
Hi,

I am running a bacula server for years now, it has been upgraded since its 2.x version and now I got it to 7.0.5+dfsg-4build1 on a Ubuntu server 16.04 with a postgresql database.

Since its upgrade to 7.x I could not yet test a restore (shame on me, I know), and now a user asked for some yesterday's files. Guess what: "For one or more of the JobIds selected, no files were found, so file selection is not possible.". That happens to all clients and filesets.

I searched Google and found some old and recent messages regarding this situation and none seems to apply to this case. Some info:

Client {
        Name            = orion-fd
        Address         = 10.0.0.1
        FDPort          = 9102
        Catalog         = CatalogoA1Sede
        Password        = "aaa"
        *File Retention  = 6 months*
        Job Retention   = 1 year
        AutoPrune       = yes
        Maximum Concurrent Jobs = 1
}

Last full job for this client was on 2016-06-01, so much less than 6 months. Yesterday's INC was like this:
15-Jun 22:17 phobos-dir JobId 39191: Start Backup JobId 39191, Job=JobOrionFab.2016-06-15_22.00.01_35
15-Jun 22:17 phobos-dir JobId 39191: Using Device "LTO" to write.
15-Jun 22:22 phobos-sd JobId 39191: Elapsed time=00:04:50, Transfer rate=52.40 M Bytes/second
15-Jun 22:22 phobos-dir JobId 39191: Bacula phobos-dir 7.0.5 (28Jul14):
  Build OS:               x86_64-pc-linux-gnu ubuntu 16.04
  JobId:                  39191
  Job:                    JobOrionFab.2016-06-15_22.00.01_35
  Backup Level:           Incremental, since=2016-06-14 22:18:51
  Client:                 "orion-fd" 5.2.5 (26Jan12) x86_64-pc-linux-gnu,ubuntu,12.04
  FileSet:                "FAB" 2016-01-04 15:27:59
  Pool:                   "PoolDiarioSemanal" (From Run Pool override)
  Catalog:                "CatalogoA1Sede" (From Client resource)
  Storage:                "LTO" (From Pool resource)
  Scheduled time:         15-Jun-2016 22:00:01
  Start time:             15-Jun-2016 22:17:30
  End time:               15-Jun-2016 22:22:21
  Elapsed time:           4 mins 51 secs
  Priority:               10  FD Files Written:       1,314
  SD Files Written:       1,314
  FD Bytes Written:       15,197,809,897 (15.19 GB)
  SD Bytes Written:       15,198,047,213 (15.19 GB)
  Rate:                   52226.2 KB/s
  Software Compression:   None
  VSS:                    no
  Encryption:             no
  Accurate:               no
  Volume name(s):         PDS-0002
  Volume Session Id:      90
  Volume Session Time:    1465823891
  Last Volume Bytes:      1,086,222,670,848 (1.086 TB)
  Non-fatal FD errors:    0
  SD Errors:              0
  FD termination status:  OK
  SD termination status:  OK
  Termination:            Backup OK

15-Jun 22:22 phobos-dir JobId 39191: Begin pruning Jobs older than 1 year .
15-Jun 22:22 phobos-dir JobId 39191: No Jobs found to prune.
15-Jun 22:22 phobos-dir JobId 39191: Begin pruning Files.
15-Jun 22:22 phobos-dir JobId 39191: No Files found to prune.
15-Jun 22:22 phobos-dir JobId 39191: End auto prune.
No files or jobs were pruned, looks good.

As per this thread http://www.mail-archive.com/bacula-users AT lists.sourceforge DOT net/msg41594.html I run those queries on db:

select jobfiles from job where jobid=39191;
Result: 1314, which matches "SD Files Written" above. Alas:

select count(*) from file where jobid=39191;
Result: 0

When upgraded from 5.x to 7.x I had to add the "Maximum Concurrent Jobs" directive to all client and job defs otherwise it would wait for max something. Perhaps I missed something else? Has the file records been pruned from db or never been there?

While dbcheck runs, can anybody kindly help me find what's wrong or what happened?

Dir conf:
#
Director {
    Name               = phobos-dir
    DIRport            = 9101                # where we listen for UA connections
    QueryFile          = "/etc/bacula/scripts/query.sql"
    WorkingDirectory   = "/var/lib/bacula"
    PidDirectory       = "/var/run/bacula"
    Maximum Concurrent Jobs = 2
    Password           = "aaa"         # Console password
    Messages           = Daemon
    # DirAddress       = 10.0.0.2
    MaximumConsoleConnections = 20
}

JobDefs {
    Name                 = "jobDiarioSemanal"
    Type                 = Backup
    Level                = Incremental
    Client               = phobos-fd
    FileSet              = Config
    Schedule             = SchDefault
    Storage              = LTO
    Pool                 = PoolDiarioSemanal
    Messages             = Standard
    Priority             = 10
    Write Bootstrap      = "/var/lib/bacula/%c.bsr"
    Allow Duplicate Jobs = no
    Cancel Lower Level Duplicates = yes
    Enabled              = yes
    Maximum Concurrent Jobs    = 1
}

JobDefs {
    Name                = "jobMensal"
    Type                = Backup
    Level               = Full
    Client              = phobos-fd
    FileSet             = Config
    Schedule            = SchMensal
    Storage             = LTO
    Pool                = PoolDiarioSemanal
    Messages            = Standard
    Priority            = 7
    Write Bootstrap     = "/var/lib/bacula/%c.bsr"
    Allow Duplicate Jobs = no
    Cancel Lower Level Duplicates = yes
    Enabled             = yes
    Maximum Concurrent Jobs = 1
}

JobDefs {
    Name                 = "jobAnual"
    Type                 = Backup
    Level                = Full
    Client               = phobos-fd
    FileSet              = Config
    Schedule             = SchAnual
    Storage              = LTO
    Pool                 = PoolAnual
    Messages             = Standard
    Priority             = 5
    Write Bootstrap      = "/var/lib/bacula/%c.bsr"
    Allow Duplicate Jobs = no
    Cancel Lower Level Duplicates = yes
    Enabled              = yes
    Maximum Concurrent Jobs = 1
}

JobDefs {
    Name                = "jobsManuais"
    Type                = Backup
    Level               = Incremental
    Client              = phobos-fd
    FileSet             = Config
    Storage             = LTO
    Pool                = PoolDiarioSemanal
    Messages            = Standard
    Priority            = 10
    Write Bootstrap     = "/var/lib/bacula/%c.bsr"
    Enabled             = yes
    Maximum Concurrent Jobs = 1
}

# Include subfiles associated with configuration of clients.
# They define the bulk of the Clients, Jobs, and FileSets.
# Remember to "reload" the Director after adding a client file.
# ClientDefs
@|"sh -c 'for f in /etc/bacula/clientdefs/*.conf ; do echo @${f} ; done'"

Job {
    Name                = "BackupCatalog"
    JobDefs             = "jobDiarioSemanal"
    Client              = phobos-fd
    Level               = Full
    FileSet             = "Catalog"
    Storage             = LTO
    Pool                = PoolDiarioSemanal
    SpoolData           = no
    Schedule            = "WeeklyCycleAfterBackup"
    RunBeforeJob        = "/etc/bacula/scripts/make_catalog_backup.pl CatalogoA1Sede"
    RunAfterJob         = "/etc/bacula/scripts/delete_catalog_backup"    # This deletes the copy of the catalog
    Write Bootstrap     = "/var/lib/bacula/%n.bsr"
    Priority            = 11                   # run after main backup
    Maximum Concurrent Jobs = 1
}
Job {
    Name            = "RestoreFiles"
    Type            = Restore
    Client          = phobos-fd
    FileSet         = Config
    Storage         = LTO
    Pool            = PoolDiarioSemanal
    Messages        = Standard
    Where           = /a1/restores
    Maximum Concurrent Jobs = 1
}

# FileSets
@|"sh -c 'for f in /etc/bacula/filesets/*.conf ; do echo @${f} ; done'"

# Schedules
Schedule {
    Name   = "SchDefault"
    Run    = Level=Incremental    Pool=PoolDiarioSemanal        daily monday-friday at 22:00
    Run    = Level=Differential    Pool=PoolDiarioSemanal        weekly sunday at 18:00
    Run    = Level=Full        Pool=PoolMensal            monthly 1 at 08:00
}
Schedule {
    Name   = "SchMensal"
    Run    = Level=Incremental    Pool=PoolDiarioSemanal        monthly 1 at 08:00
}
Schedule {
    Name   = "SchAnual"
    Run    = Level=Full        Pool=PoolAnual        jan 1 at 08:00
}
Schedule {
    Name   = "WeeklyCycleAfterBackup"
    Run    = Full mon-sat at 03:00
}
Schedule {
    Name   = "SchDesativado"
}

# This is the backup of the catalog
FileSet {
    Name    = "Catalog"
    Include {
      Options {
        signature    = MD5
      }
      File = "/a1/database/bacula"
    }
}
Catalog {
    Name    = CatalogoA1Sede
    dbname  = bacula ; DB Address = "127.0.0.1"; dbuser = "bacula"; dbpassword = "aaa"
}
Messages {
    Name              = Standard
    mailcommand       = "/usr/lib/bacula/bsmtp -h localhost -f \"\(Bacula\) \<%r\>\" -s \"Bacula: %t %l %e %n of %c\" %r"
    operatorcommand   = "/usr/lib/bacula/bsmtp -h localhost -f \"\(Bacula\) \<%r\>\" -s \"Bacula: Intervention needed for %j\" %r"
    mail              = backup    = all, !skipped           
    operator          = backup    = mount
    console           = all, !skipped, !saved
    MailOnError       = backup = all
    append            = "/var/lib/bacula/log"    = all, !skipped
    catalog           = all
}
Messages {
    Name           = Daemon
    mailcommand    = "/usr/lib/bacula/bsmtp -h localhost -f \"\(Bacula\) \<%r\>\" -s \"Bacula daemon message\" %r"
    mail           = noc AT a1.ind DOT br    = all, !skipped           
    console        = all, !skipped, !saved
    append         = "/var/lib/bacula/log"    = all, !skipped
}


# Storages
Storage {
    Name        = LTO
    Address     = 10.0.0.2
    SDPort      = 9103
    Password    = "aaa"
    Device      = LTO
    Media Type  = LTO-4
    Autochanger = no
    Maximum Concurrent Jobs = 1
}

# Pools
Pool {
    Name             = PoolDiarioSemanal
    Pool Type        = Backup
    Recycle          = yes
    AutoPrune        = yes
    Volume Retention = 30 days
    LabelFormat      = "PDS-"
    Storage          = LTO
}

Pool {
    Name             = PoolMensal
    Pool Type        = Backup
    Recycle          = yes
    AutoPrune        = yes
    Volume Retention = 365 days
    LabelFormat      = "PMF-"
    Storage          = LTO
}
Pool {
    Name             = PoolAnual
    Pool Type        = Backup
    Recycle          = no
    AutoPrune        = no
    Volume Retention = 10 years
    LabelFormat      = "PAF-"
    Storage          = LTO
}
Pool {
    Name        = Scratch
    Pool Type   = Backup
}
Console {
    Name        = phobos-mon
    Password    = "aaa"
    CommandACL  = status, .status
}

Thanks in advance, best regards.


--
Marcio Merlone

------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity planning
reports. http://sdm.link/zohomanageengine
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users


------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
[More with this subject...]
<Prev in Thread] Current Thread [Next in Thread>
Previous by Date:  Re: [Bacula-users] [Questions about Bacular Backup solution], Ana Emília M. Arruda
Next by Date:  Re: [Bacula-users] get the size of all backup jobs for client, Ana Emília M. Arruda
Previous by Thread:  [Bacula-users] Files pruned from catalog or never inserted - For one or more of the JobIds selected, no files were found, so file selection is not possible., Marcio Vogel Merlone dos Santos
Next by Thread:  [Bacula-users] bacula waiting for a mount request during restore, Daniel
Indexes:  [Date] [Thread] [Top] [All Lists]