Bacula-users

Re: [Bacula-users] Client backup with compress

2016-03-27 05:11:26
Subject: Re: [Bacula-users] Client backup with compress
From: Kern Sibbald <kern AT sibbald DOT com>
To: Randy Katz <rkatz AT simplicityhosting DOT com>, "More, Ankush" <ankush.more AT capgemini DOT com>
Date: Sun, 27 Mar 2016 19:06:53 +1000
In the output there was an error from the FD when debug information was 
turned on. It pertained to the Exclude = xxx that was in that particular 
email.

The FileSet shown below looks perfectly good to me.  Note: in some older 
version, there was a problem with compression when both libgzip and 
liblzo2 and their development versions were not loaded on the machine 
used to build the FD.  This is currently fixed, so I would recommend you 
move up to 7.4.x if you can, but that is not always so simple.

Best regards,
Kern

On 03/27/2016 10:28 AM, Randy Katz wrote:
> Kern, if you have been following this my fileset is quite default, does
> not receive an error, and still does not compress,
> only on one client out of about 40:
>
> FileSet {
>    Name = "client-28-filesetdefinition"
>    Include {
>      Options {
>        signature = MD5
>        compression = GZIP
>      }
>      File = /
>    }
>    Exclude {
>      File = /tmp
>      File = /proc
>      File = /.journal
>      File = /.fsck
>      File = /home/virtfs
>    }
> }
>
> On 3/26/2016 5:09 PM, Kern Sibbald wrote:
>> Note, the minute you get a message like:
>>
>>   bacula-fd: job.c:781-415 Set state=error or double code
>>
>> all bets are off. I am not sure what the code will do, but Bacula
>> thinks you have an error in your fileset, so until you correct it, any
>> other problems could be a result of this.
>>
>> Best regards,
>> Kern
>>
>>
>> On 03/25/2016 06:45 AM, More, Ankush wrote:
>>> Hello Randy,
>>>
>>> Bacula server(7.x) is installed with Linux RPM package.
>>>
>>> Catalog backup on bacula server work with compression = GZIP option.
>>>
>>> Job log:
>>>
>>> Software Compression:   84.9% 6.6:1
>>>
>>> But  clients  does not work
>>>
>>> Job log:
>>>
>>> Software Compression:   None
>>>
>>> Thank you,
>>>
>>> Ankush More
>>>
>>> *From:*Randy Katz [mailto:rkatz AT simplicityhosting DOT com]
>>> *Sent:* Thursday, March 24, 2016 8:10 PM
>>> *To:* More, Ankush
>>> *Subject:* Re: [Bacula-users] Client backup with compress
>>>
>>> Then the only other thing I could think of is if the director is
>>> compiled with
>>> the gzip/lzo libraries, have you recompiled that? Should not be too
>>> hard, you don't
>>> need that long configure line, if using mysql all you need is:
>>>
>>> ./configure --with-mysql
>>>
>>>
>>> On 3/24/2016 6:54 AM, More, Ankush wrote:
>>>
>>>      Hi Randy,
>>>
>>>      Backing up are Database files which has very high compression ratio.
>>>
>>>      I have check same files with  linux command ‘gzip’ on client working
>>>      fine with good compression ratio.
>>>
>>>      But Bacula does not work.
>>>
>>>      Thank you,
>>>
>>>      Ankush More
>>>
>>>      *From:*Randy Katz [mailto:rkatz AT simplicityhosting DOT com]
>>>      *Sent:* Thursday, March 24, 2016 1:18 PM
>>>      *To:* bacula-users AT lists.sourceforge DOT net
>>>      <mailto:bacula-users AT lists.sourceforge DOT net>
>>>      *Subject:* Re: [Bacula-users] Client backup with compress
>>>
>>>      Hi Ankush, are you sure the files you are backing up can be
>>>      compressed? You might
>>>      want to try a separate backup on the same client of something like
>>>      /usr/local /etc
>>>      just to test, these should definitely compress. Sometimes video,
>>>      audio and other dense
>>>      files do not compress.
>>>
>>>      On 3/23/2016 11:44 AM, More, Ankush wrote:
>>>
>>>          Hello,
>>>
>>>          I re-compile client with LZO package.
>>>
>>>          Compression does not work both GZIP or LZO.
>>>
>>>              Large file support:      yes
>>>
>>>              Bacula conio support:    yes -lncurses
>>>
>>>              readline support:        no
>>>
>>>              TCP Wrappers support:    no
>>>
>>>              TLS support:             yes
>>>
>>>              Encryption support:      yes
>>>
>>>          ZLIB support:            yes
>>>
>>>              LZO support:             yes
>>>
>>>              enable-smartalloc:       yes
>>>
>>>              enable-lockmgr:          no
>>>
>>>              bat support:             no
>>>
>>>              client-only:             yes
>>>
>>>              build-dird:              no
>>>
>>>              build-stored:            no
>>>
>>>              Plugin support:          yes
>>>
>>>              AFS support:             no
>>>
>>>              ACL support:             yes
>>>
>>>              XATTR support:           yes
>>>
>>>              systemd support:         no
>>>
>>>              Batch insert enabled:    None
>>>
>>>          Thank you,
>>>
>>>          Ankush More
>>>
>>>          *From:*More, Ankush
>>>          *Sent:* Wednesday, March 23, 2016 8:46 PM
>>>          *To:* 'Heitor Faria'
>>>          *Cc:* compdoc AT hotrodpc DOT com <mailto:compdoc AT hotrodpc DOT 
>>> com>;
>>>          bacula-users AT lists.sourceforge DOT net
>>>          <mailto:bacula-users AT lists.sourceforge DOT net>
>>>          *Subject:* RE: [Bacula-users] Client backup with compress
>>>
>>>          Hello Faria,
>>>
>>>          Sorry for confusion , please ignore previous mail. I have
>>>          re-compile the client with source which version 7.x on Linux.
>>>
>>>          Resending debug –d500 output for same.
>>>
>>>          Which support ZLIB, but still compression is not working.
>>>
>>>          ZLIB support:            yes
>>>
>>>          Thank you,
>>>
>>>          Ankush More
>>>
>>>          *From:*Heitor Faria [mailto:heitor AT bacula.com DOT br]
>>>          *Sent:* Wednesday, March 23, 2016 8:31 PM
>>>          *To:* More, Ankush
>>>          *Cc:* compdoc AT hotrodpc DOT com <mailto:compdoc AT hotrodpc DOT 
>>> com>;
>>>          bacula-users AT lists.sourceforge DOT net
>>>          <mailto:bacula-users AT lists.sourceforge DOT net>
>>>          *Subject:* Re: [Bacula-users] Client backup with compress
>>>
>>>          Hello, Akush: according to the debug you sent me this is the
>>>          FileSet you are using for this job:
>>>
>>>          bacula-fd: job.c:273-415 Executing fileset command.
>>>          bacula-fd: job.c:753-415 I
>>>          bacula-fd: job.c:753-415 O MZ6
>>>          bacula-fd: job.c:753-415 N
>>>          bacula-fd: job.c:753-415 O e
>>>          bacula-fd: job.c:753-415 WF *.apm
>>>          bacula-fd: job.c:781-415 Set state=error or double code.
>>>          bacula-fd: job.c:753-415 N
>>>          bacula-fd: job.c:753-415 F /oracle
>>>          bacula-fd: job.c:753-415 N
>>>          bacula-fd: job.c:753-415 N
>>>          bacula-fd: job.c:753-415 E
>>>          bacula-fd: job.c:753-415 F /var/spool/bacula
>>>          bacula-fd: job.c:753-415 F /tmp
>>>          bacula-fd: job.c:753-415 F /proc
>>>          bacula-fd: job.c:753-415 F /tmp
>>>          bacula-fd: job.c:753-415 F /sys
>>>          bacula-fd: job.c:753-415 F /.journal
>>>          bacula-fd: job.c:753-415 F /.fsck
>>>          bacula-fd: job.c:753-415 N
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>>              *From: *"More, Ankush" <ankush.more AT capgemini DOT com
>>>              <mailto:ankush.more AT capgemini DOT com>>
>>>              *To: *"Heitor Faria" <heitor AT bacula.com DOT br
>>>              <mailto:heitor AT bacula.com DOT br>>
>>>              *Cc: *compdoc AT hotrodpc DOT com <mailto:compdoc AT hotrodpc 
>>> DOT com>,
>>>              bacula-users AT lists.sourceforge DOT net
>>>              <mailto:bacula-users AT lists.sourceforge DOT net>
>>>              *Sent: *Wednesday, March 23, 2016 11:57:12 AM
>>>              *Subject: *RE: [Bacula-users] Client backup with compress
>>>
>>>              Hello Faria,
>>>
>>>              Below is DIR config file.
>>>
>>>              FileSet {
>>>
>>>                 Name = "chrome02 Set"
>>>
>>>                 Include {
>>>
>>>                   Options {
>>>
>>>                     signature = MD5
>>>
>>>                       compression = GZIP
>>>
>>>                   }
>>>
>>>              Not sure what you mean Compression=gzip in all of them.
>>>
>>>              Thank you,
>>>
>>>              Ankush More
>>>
>>>              *From:*Heitor Faria [mailto:heitor AT bacula.com DOT br]
>>>              *Sent:* Wednesday, March 23, 2016 8:14 PM
>>>              *To:* More, Ankush
>>>              *Cc:* compdoc AT hotrodpc DOT com <mailto:compdoc AT hotrodpc 
>>> DOT com>;
>>>              bacula-users AT lists.sourceforge DOT net
>>>              <mailto:bacula-users AT lists.sourceforge DOT net>
>>>              *Subject:* Re: [Bacula-users] Client backup with compress
>>>
>>>              Hello, Ankush:   ZLIB support:            yes
>>>
>>>              Your client is fine. It must be a FileSet configuration
>>>              thing, since you seems to have multiple Options
>>>              sub-resources. Just put Compression=gzip in all of them.
>>>
>>>              Regards,
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>>                  *From: *"More, Ankush" <ankush.more AT capgemini DOT com
>>>                  <mailto:ankush.more AT capgemini DOT com>>
>>>                  *To: *compdoc AT hotrodpc DOT com
>>>                  <mailto:compdoc AT hotrodpc DOT com>,
>>>                  bacula-users AT lists.sourceforge DOT net
>>>                  <mailto:bacula-users AT lists.sourceforge DOT net>
>>>                  *Sent: *Wednesday, March 23, 2016 11:32:34 AM
>>>                  *Subject: *Re: [Bacula-users] Client backup with
>>> compress
>>>
>>>                  Hello,
>>>
>>>                  Please see below, this is build for bacula client.
>>>
>>>                  Configuration on Wed Mar 23 04:37:28 EDT 2016:
>>>
>>>                      Host:                    x86_64-unknown-linux-gnu --
>>>                  redhat Enterprise release
>>>
>>>                      Bacula version:          Bacula 7.0.4 (04 June 2014)
>>>
>>>                      Source code location:    .
>>>
>>>                      Install binaries:        /usr/sbin
>>>
>>>                      Install libraries:       /usr/lib64
>>>
>>>                      Install config files:    /etc/bacula
>>>
>>>                      Scripts directory:       /usr/libexec/bacula/
>>>
>>>                      Archive directory:       /tmp
>>>
>>>                      Working directory:       /var/spool/bacula/
>>>
>>>                      PID directory:           /var/run
>>>
>>>                      Subsys directory:        /var/lock/subsys
>>>
>>>                      Man directory:           ${datarootdir}/man
>>>
>>>                      Data directory:          /usr/share
>>>
>>>                      Plugin directory:        /usr/lib64
>>>
>>>                      C Compiler:              gcc 4.1.2
>>>
>>>                      C++ Compiler:            /usr/bin/g++ 4.1.2
>>>
>>>                      Compiler flags:           -g -O2 -Wall
>>>                  -fno-strict-aliasing -fno-exceptions -fno-rtti
>>>
>>>                      Linker flags:
>>>
>>>                      Libraries:               -lpthread -ldl -ldl
>>>
>>>                      Statically Linked Tools: no
>>>
>>>                      Statically Linked FD:    no
>>>
>>>                      Statically Linked SD:    no
>>>
>>>                      Statically Linked DIR:   no
>>>
>>>                      Statically Linked CONS:  no
>>>
>>>                      Database backends:       None
>>>
>>>                      Database port:
>>>
>>>                      Database name:           bacula
>>>
>>>                      Database user:           bacula
>>>
>>>                      Job Output Email:        root@localhost
>>>
>>>                      Traceback Email:         root@localhost
>>>
>>>                      SMTP Host Address:       localhost
>>>
>>>                      Director Port:           9101
>>>
>>>                      File daemon Port:        9102
>>>
>>>                      Storage daemon Port:     9103
>>>
>>>                      Director User:
>>>
>>>                      Director Group:
>>>
>>>                      Storage Daemon User:
>>>
>>>                      Storage DaemonGroup:
>>>
>>>                      File Daemon User:
>>>
>>>                      File Daemon Group:
>>>
>>>                      Large file support:      yes
>>>
>>>                      Bacula conio support:    yes -lncurses
>>>
>>>                      readline support:        no
>>>
>>>                      TCP Wrappers support:    no
>>>
>>>                      TLS support:             yes
>>>
>>>                      Encryption support:      yes
>>>
>>>                      ZLIB support:            yes
>>>
>>>                      LZO support:             no
>>>
>>>                      enable-smartalloc:       yes
>>>
>>>                      enable-lockmgr:          no
>>>
>>>                      bat support:             no
>>>
>>>                      client-only:             yes
>>>
>>>                      build-dird:              no
>>>
>>>                      build-stored:            no
>>>
>>>                      Plugin support:          yes
>>>
>>>                      AFS support:             no
>>>
>>>                      ACL support:             yes
>>>
>>>                      XATTR support:           yes
>>>
>>>                      systemd support:         no
>>>
>>>                      Batch insert enabled:    None
>>>
>>>                  Thank you,
>>>
>>>                  Ankush More
>>>
>>>                  *From:*compdoc [mailto:compdoc AT hotrodpc DOT com]
>>>                  *Sent:* Wednesday, March 23, 2016 7:48 PM
>>>                  *To:* bacula-users AT lists.sourceforge DOT net
>>>                  <mailto:bacula-users AT lists.sourceforge DOT net>
>>>                  *Subject:* Re: [Bacula-users] Client backup with
>>> compress
>>>
>>>                  >#./configure --enable-client-only
>>> --enable-build-dird=no --enable-build-stored=no --bindir=/usr/bin
>>> --sbindir=/usr/sbin --with-scriptdir=/usr/libexec/bacula/
>>> --with-working-dir=/var/spool/bacula/
>>>                  --with-logdir=/var/log --enable-smartalloc
>>>
>>>                  But where is the output from this configure? It will say
>>>                  if compression is available at the end. Bacula places a
>>>                  file named bacula_config in your scripts directory that
>>>                  contains the output:
>>>
>>>                  Configuration on Sat Feb 13 16:32:43 MST 2016:
>>>
>>>                      Host:
>>>                  x86_64-unknown-linux-gnu -- ubuntu 14.04
>>>
>>>                      Bacula version:         Bacula 7.4.0 (16 January
>>> 2016)
>>>
>>>                      Source code location:     .
>>>
>>>                     Install binaries:          /sbin
>>>
>>>                      Install libraries:          /usr/lib64
>>>
>>>                      Install config files:     /etc/bacula
>>>
>>>                      Scripts directory:       /etc/bacula/scripts
>>>
>>>                      Archive directory:      /tmp
>>>
>>>                      Working directory:
>>>                  /opt/bacula/working
>>>
>>>                      PID directory:            /var/run
>>>
>>>                      Subsys directory:      /var/run/subsys
>>>
>>>                      Man directory:           ${datarootdir}/man
>>>
>>>                      Data directory:          /usr/share
>>>
>>>                      Plugin directory:        /usr/lib64
>>>
>>>                      C Compiler:                              gcc
>>>                  4.8.4-2ubuntu1~14.04)
>>>
>>>                      C++ Compiler:           /usr/bin/g++
>>>                  4.8.4-2ubuntu1~14.04)
>>>
>>>                      Compiler flags:          -O2 -g -Wall -x c++
>>>                  -fno-strict-aliasing -fno-exceptions -fno-rtti
>>>
>>>                      Linker flags:
>>>
>>>                      Libraries:
>>>                  -lpthread -ldl -ldl
>>>
>>>                      Statically Linked Tools:  no
>>>
>>>                      Statically Linked FD:     no
>>>
>>>                      Statically Linked SD:     no
>>>
>>>                      Statically Linked DIR:    no
>>>
>>>                      Statically Linked CONS:   no
>>>
>>>                      Database backends:                MySQL
>>>
>>>                      Database port:
>>>
>>>                      Database name:        bacula
>>>
>>>                      Database user:          bacula
>>>
>>>                      Database SSL options:
>>>
>>>                      Job Output Email: compdoc AT domain DOT com
>>>                  <mailto:compdoc AT domain DOT com>
>>>
>>>                      Traceback Email: compdoc AT domain DOT com
>>>                  <mailto:compdoc AT domain DOT com>
>>>
>>>                      SMTP Host Address:                mail.domain.com
>>>
>>>                      Director Port:            9101
>>>
>>>                      File daemon Port:      9102
>>>
>>>                      Storage daemon Port:      9103
>>>
>>>                      Director User:
>>>
>>>                      Director Group:
>>>
>>>                      Storage Daemon User:
>>>
>>>                      Storage DaemonGroup:
>>>
>>>                      File Daemon User:
>>>
>>>                      File Daemon Group:
>>>
>>>                      Large file support:                    yes
>>>
>>>                      Bacula conio support:     no -lreadline -ltinfo
>>>
>>>                      readline support:       yes
>>>
>>>                      TCP Wrappers support:     no
>>>
>>>                      TLS support:             yes
>>>
>>>                      Encryption support:                  yes
>>>
>>>                      ZLIB support:            yes
>>>
>>>                      LZO support:            yes
>>>
>>>                      enable-smartalloc:                    yes
>>>
>>>                      enable-lockmgr:        no
>>>
>>>                      bat support:              no
>>>
>>>                      client-only:                no
>>>
>>>                      build-dird:                                yes
>>>
>>>                      build-stored:             yes
>>>
>>>                      Plugin support:         yes
>>>
>>>                      AFS support:            no
>>>
>>>                      ACL support:            yes
>>>
>>>                      XATTR support:        yes
>>>
>>>                      systemd support:      no
>>>
>>>                      Batch insert enabled:     MySQL
>>>
>>>                  This message contains information that may be privileged
>>>                  or confidential and is the property of the Capgemini
>>>                  Group. It is intended only for the person to whom it is
>>>                  addressed. If you are not the intended recipient, you
>>>                  are not authorized to read, print, retain, copy,
>>>                  disseminate, distribute, or use this message or any part
>>>                  thereof. If you receive this message in error, please
>>>                  notify the sender immediately and delete all copies of
>>>                  this message.
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>>                  Transform Data into Opportunity.
>>>                  Accelerate data analysis in your applications with
>>>                  Intel Data Analytics Acceleration Library.
>>>                  Click to learn more.
>>>
>>> http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140
>>>                  _______________________________________________
>>>                  Bacula-users mailing list
>>>                  Bacula-users AT lists.sourceforge DOT net
>>>                  <mailto:Bacula-users AT lists.sourceforge DOT net>
>>>
>>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>>
>>>              --
>>>
>>>
>>> =======================================================================
>>>              Heitor Medrado de Faria  - LPIC-III | ITIL-F |  Bacula
>>>              Systems Certified Administrator II
>>>
>>>              Próximas aulas telepresencial ao-vivo - 15 de fevereiro:
>>>              http://www.bacula.com.br/agenda/
>>>
>>>              Ministro treinamento e implementação in-company Bacula:
>>>              http://www.bacula.com.br/in-company/
>>>
>>>              Ou assista minhas videoaulas on-line:
>>>              http://www.bacula.com.br/treinamento-bacula-ed/
>>>
>>>              61 <tel:%2B55%2061%202021-8260>8268-4220
>>>              <tel:%2B55%2061%208268-4220>
>>>
>>>              Site: www.bacula.com.br
>>>              <http://www.bacula.com.br> | Facebook: heitor.faria
>>>              <http://www.facebook.com/heitor.faria>
>>>
>>>
>>> ========================================================================
>>>
>>>          --
>>>
>>>
>>> =======================================================================
>>>          Heitor Medrado de Faria  - LPIC-III | ITIL-F |  Bacula Systems
>>>          Certified Administrator II
>>>
>>>          Próximas aulas telepresencial ao-vivo - 15 de fevereiro:
>>>          http://www.bacula.com.br/agenda/
>>>
>>>          Ministro treinamento e implementação in-company Bacula:
>>>          http://www.bacula.com.br/in-company/
>>>
>>>          Ou assista minhas videoaulas on-line:
>>>          http://www.bacula.com.br/treinamento-bacula-ed/
>>>
>>>          61 <tel:%2B55%2061%202021-8260>8268-4220
>>>          <tel:%2B55%2061%208268-4220>
>>>
>>>          Site: www.bacula.com.br <http://www.bacula.com.br> | Facebook:
>>>          heitor.faria <http://www.facebook.com/heitor.faria>
>>>
>>>
>>> ========================================================================
>>>
>>>
>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>>
>>>          Transform Data into Opportunity.
>>>
>>>          Accelerate data analysis in your applications with
>>>
>>>          Intel Data Analytics Acceleration Library.
>>>
>>>          Click to learn more.
>>>
>>>          http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140
>>>
>>>
>>>
>>>
>>>
>>>          _______________________________________________
>>>
>>>          Bacula-users mailing list
>>>
>>>          Bacula-users AT lists.sourceforge DOT net
>>>          <mailto:Bacula-users AT lists.sourceforge DOT net>
>>>
>>>          https://lists.sourceforge.net/lists/listinfo/bacula-users
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>>
>>> Transform Data into Opportunity.
>>> Accelerate data analysis in your applications with
>>> Intel Data Analytics Acceleration Library.
>>> Click to learn more.
>>> http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140
>>>
>>>
>>>
>>> _______________________________________________
>>> Bacula-users mailing list
>>> Bacula-users AT lists.sourceforge DOT net
>>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>>
>>
>

------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

<Prev in Thread] Current Thread [Next in Thread>