Hello Ankush,

There is a good explanation about the Options resource of FileSets in http://www.bacula.org/7.4.x-manuals/en/main/main.pdf on page 166. After reading it carefully I could understand why compression was not working for you (specifically, the FileSet you posted). I recommend you have a look at it :).

Best regards,

Ana

On Wed, Mar 30, 2016 at 12:06 AM, Ana Emília M. Arruda <emiliaarruda AT gmail DOT com> wrote:

Hello Ankush,

You're welcome.

compression = GZIP should work for Windows.

Best regards,

Ana

On Mon, Mar 28, 2016 at 7:06 AM, More, Ankush <ankush.more AT capgemini DOT com> wrote:

Hello Ana,

 

Thanks for your effort to resolve issue.

FileSet which you have send work fine for software compress.

 

What compression command (GZIP,LZO etc) use for windows client?

 

compression = GZIP ( what for windows client)

 

 

Thank you,

Ankush More

 

From: Ana Emília M. Arruda [mailto:emiliaarruda AT gmail DOT com]
Sent: Saturday, March 26, 2016 6:47 AM
To: Randy Katz
Cc: Bacula-users AT lists.sourceforge DOT net

Subject: Re: [Bacula-users] Client backup with compress

 

Hello,

I'm sorry, please disregard my last post (the two filesets are identical). Following is a more detailed test (and this is not a cosmetic error of the backup log):

1) compression works.

FileSet {
  Name = "Full Set"
  Include {
    Options {
       exclude = yes
       wildfile = "*.data*"
    }
    Options {
      signature = MD5
      compression = GZIP
    }
    File = /opt/bacula/bin
  }

  Build OS:               x86_64-unknown-linux-gnu ubuntu 14.04
  JobId:                  62
  Job:                    BackupClient1.2016-03-26_01.37.17_04
  Backup Level:           Full
  Client:                 "bacula-server-fd" 7.4.0 (16Jan16) x86_64-unknown-linux-gnu,ubuntu,14.04
  FileSet:                "Full Set" 2016-02-19 23:35:17
...
  FD Files Written:       17
  SD Files Written:       17
  FD Bytes Written:       7,015,847 (7.015 MB)
  SD Bytes Written:       7,017,616 (7.017 MB)
  Rate:                   3507.9 KB/s
  Software Compression:   58.0% 2.4:1
...
  Volume name(s):         Vol-0001
  Volume Session Id:      1
  Volume Session Time:    1458952340
  Last Volume Bytes:      7,025,678 (7.025 MB)
...
  Termination:            Backup OK

-rw-r----- 1 root root  7025678 mar 26 01:37 Vol-0001

2) compression do not work:

FileSet {
  Name = "Full Set"
  Include {
    Options {
      signature = MD5
      compression = GZIP
    }
    Options {
       exclude = yes
       wildfile = "*.data*"
    }
    File = /opt/bacula/bin
  }

  Build OS:               x86_64-unknown-linux-gnu ubuntu 14.04
  JobId:                  63
  Job:                    BackupClient1.2016-03-26_01.41.46_03
  Backup Level:           Full
  Client:                 "bacula-server-fd" 7.4.0 (16Jan16) x86_64-unknown-linux-gnu,ubuntu,14.04
  FileSet:                "Full Set" 2016-02-19 23:35:17
...
  FD Files Written:       17
  SD Files Written:       17
  FD Bytes Written:       16,691,953 (16.69 MB)
  SD Bytes Written:       16,693,466 (16.69 MB)
  Rate:                   16692.0 KB/s
  Software Compression:   None
...
  Volume name(s):         Vol-0001
  Volume Session Id:      1
  Volume Session Time:    1458952842
  Last Volume Bytes:      16,706,736 (16.70 MB)
...
  Termination:            Backup OK

-rw-r----- 1 root root 16706736 mar 26 01:41 Vol-0001

3) compression works:
FileSet {
  Name = "Full Set"
  Include {
    Options {
      signature = MD5
      compression = GZIP
    }
    Options {
       exclude = yes
       compression = GZIP
       wildfile = "*.data*"
    }
    File = /opt/bacula/bin
  }

  Build OS:               x86_64-unknown-linux-gnu ubuntu 14.04
  JobId:                  65
  Job:                    BackupClient1.2016-03-26_01.50.53_03
  Backup Level:           Full
  Client:                 "bacula-server-fd" 7.4.0 (16Jan16) x86_64-unknown-linux-gnu,ubuntu,14.04
  FileSet:                "Full Set" 2016-02-19 23:35:17
...
  FD Files Written:       17
  SD Files Written:       17
  FD Bytes Written:       7,015,847 (7.015 MB)
  SD Bytes Written:       7,017,360 (7.017 MB)
  Rate:                   3507.9 KB/s
  Software Compression:   58.0% 2.4:1
...
  Volume name(s):         Vol-0001
  Volume Session Id:      1
  Volume Session Time:    1458953395
  Last Volume Bytes:      7,025,230 (7.025 MB)
...
  Termination:            Backup OK

-rw-r----- 1 root root  7025230 mar 26 01:50 Vol-0001

Notes:

- I created some *.data* files in /opt/bacula/bin for testing purposes

- all three Filesets correctly exclude the *.data* files

- I cannot see the need of a compressed clause in the options with the exclude option set to yes. But as Heitor suggested, it works.

Randy, I have not run tests with your Fileset configuration. We should have a close look in bacula client version, linux version, zlib devel version, maybe verify carefully if there is really no difference in Fileset configurations, check if without the exclude clause compression works and finally check if the selected files for this "compression not working client" can be compressed.

 

Best regards,

Ana

 

On Sat, Mar 26, 2016 at 12:44 AM, Randy Katz <rkatz AT simplicityhosting DOT com> wrote:

My expressions are default, here is mine:

FileSet {
  Name = "client-28-filesetdefinition"
  Include {
    Options {
      signature = MD5
      compression = GZIP
    }
    File = /
  }
  Exclude {
    File = /tmp
    File = /proc
    File = /.journal
    File = /.fsck
    File = /home/virtfs

  }
}

On 3/25/2016 3:38 PM, Ana Emília M. Arruda wrote:

Hello Ankush,

The _expression_ works correctly but for some reason, the following occurs:

1) Software compression none, FD and SD bytes written shows correct values with the bellow configuration:

FileSet {
  Name = "chrome02 Set"
  Include {
     Options {
        exclude = yes
        wildfile  = "*.data*"
     }
     Options {
        signature = MD5
        compression = GZIP
      }
    File = /rman
  }


2) Software compression shows correct value for compression, FD and SD bytes written shows incorrect values with the bellow configuration:

FileSet {
  Name = "chrome02 Set"
  Include {
     Options {
        exclude = yes
        wildfile  = "*.data*"
     }
     Options {
        signature = MD5
        compression = GZIP
      }
    File = /rman
  }

 

The estimate listing command shows that both configurations work correctly. Maybe this is a cosmetic error.

 

Best regards,

Ana

 

 

On Fri, Mar 25, 2016 at 6:21 PM, Ana Emília M. Arruda <emiliaarruda AT gmail DOT com> wrote:

Hello Ankush and Randy,

 

It seems that there is a problem with the use of the wildfile _expression_. Regardless the "*.data*" is correct (tested with bwild utility), it seems that when this _expression_ is used, the backup do not work correctly and the result shows a "none" software compression.

 

bacula-fd: job.c:1646-0 Fileset: WF *.data*

bacula-fd: job.c:1252-0 WF *.data*

bacula-fd: job.c:1280-0 Set state=error or double code.

 

Could you please try this filset without the exclude option that excludes "*.data*" (the same for Randy, I don't know if your client problem uses a fileset like this one).

 

Best regards,

Ana

 

On Thu, Mar 24, 2016 at 9:18 PM, Randy Katz <rkatz AT simplicityhosting DOT com> wrote:

Yes, her client compilation confirmed compression for gzip and lzo...so now I would think there is an issue here. I also have one client out of around 40 that is not compressing. Dir and fd compiled from source version 7.4.0.

On March 25, 2016 5:29:20 AM GMT+07:00, "Ana Emília M. Arruda" <emiliaarruda AT gmail DOT com> wrote:

Hello Ankush,

 

As Heitor said in a previous post, zlib development libraries must also be installed on clients. Have you checked this? Compression is working on your bacula server (and client) for your catalog backup because the zlib development libraries are installed on the bacula director host. Sorry if you have already answered this, but I did not found in previous posts.

 

Best regards,

Ana

 

On Thu, Mar 24, 2016 at 5:45 PM, More, Ankush <ankush.more AT capgemini DOT com> wrote:

Hello Randy,

Bacula server(7.x) is installed with Linux RPM package.

Catalog backup on bacula server work with compression = GZIP option.

Job log:

Software Compression:   84.9% 6.6:1

But  clients  does not work  

Job log:

Software Compression:   None

 

 

Thank you,

Ankush More

 

From: Randy Katz [mailto:rkatz AT simplicityhosting DOT com]
Sent: Thursday, March 24, 2016 8:10 PM
To: More, Ankush
Subject: Re: [Bacula-users] Client backup with compress

 

Then the only other thing I could think of is if the director is compiled with
the gzip/lzo libraries, have you recompiled that? Should not be too hard, you don't
need that long configure line, if using mysql all you need is:

./configure --with-mysql

On 3/24/2016 6:54 AM, More, Ankush wrote:

Hi Randy,

 

Backing up are Database files which has very high compression ratio.

I have check same files with  linux command ‘gzip’ on client working fine with good compression ratio.

But Bacula does not work.

 

Thank you,

Ankush More

 

From: Randy Katz [mailto:rkatz AT simplicityhosting DOT com]
Sent: Thursday, March 24, 2016 1:18 PM
To: bacula-users AT lists.sourceforge DOT net
Subject: Re: [Bacula-users] Client backup with compress

 

Hi Ankush, are you sure the files you are backing up can be compressed? You might
want to try a separate backup on the same client of something like /usr/local /etc
just to test, these should definitely compress. Sometimes video, audio and other dense
files do not compress.

On 3/23/2016 11:44 AM, More, Ankush wrote:

Hello,

 

I re-compile client with LZO package.

Compression does not work both GZIP or LZO.

 

   Large file support:      yes

   Bacula conio support:    yes -lncurses

   readline support:        no

   TCP Wrappers support:    no

   TLS support:             yes

   Encryption support:      yes

   ZLIB support:            yes

   LZO support:             yes

   enable-smartalloc:       yes

   enable-lockmgr:          no

   bat support:             no

   client-only:             yes

   build-dird:              no

   build-stored:            no

   Plugin support:          yes

   AFS support:             no

   ACL support:             yes

   XATTR support:           yes

   systemd support:         no

   Batch insert enabled:    None

 

Thank you,

Ankush More

 

From: More, Ankush
Sent: Wednesday, March 23, 2016 8:46 PM
To: 'Heitor Faria'
Cc: compdoc AT hotrodpc DOT com; bacula-users AT lists.sourceforge DOT net
Subject: RE: [Bacula-users] Client backup with compress

 

Hello Faria,

 

Sorry for confusion , please ignore previous mail. I have re-compile the client with source which version 7.x on Linux.

Resending debug –d500 output for same.

 

Which support ZLIB, but still compression is not working.

 

ZLIB support:            yes

 

 

Thank you,

Ankush More

 

From: Heitor Faria [mailto:heitor AT bacula.com DOT br]
Sent: Wednesday, March 23, 2016 8:31 PM
To: More, Ankush
Cc: compdoc AT hotrodpc DOT com; bacula-users AT lists.sourceforge DOT net
Subject: Re: [Bacula-users] Client backup with compress

 

Hello, Akush: according to the debug you sent me this is the FileSet you are using for this job:

 

bacula-fd: job.c:273-415 Executing fileset command.
bacula-fd: job.c:753-415 I
bacula-fd: job.c:753-415 O MZ6
bacula-fd: job.c:753-415 N
bacula-fd: job.c:753-415 O e
bacula-fd: job.c:753-415 WF *.apm
bacula-fd: job.c:781-415 Set state=error or double code.
bacula-fd: job.c:753-415 N
bacula-fd: job.c:753-415 F /oracle
bacula-fd: job.c:753-415 N
bacula-fd: job.c:753-415 N
bacula-fd: job.c:753-415 E
bacula-fd: job.c:753-415 F /var/spool/bacula
bacula-fd: job.c:753-415 F /tmp
bacula-fd: job.c:753-415 F /proc
bacula-fd: job.c:753-415 F /tmp
bacula-fd: job.c:753-415 F /sys
bacula-fd: job.c:753-415 F /.journal
bacula-fd: job.c:753-415 F /.fsck
bacula-fd: job.c:753-415 N

 

---

From: "More, Ankush" <ankush.more AT capgemini DOT com>
To: "Heitor Faria" <heitor AT bacula.com DOT br>
Cc: compdoc AT hotrodpc DOT com, bacula-users AT lists.sourceforge DOT net
Sent: Wednesday, March 23, 2016 11:57:12 AM
Subject: RE: [Bacula-users] Client backup with compress

Hello Faria,

 

Below is DIR config file.

 

FileSet {

  Name = "chrome02 Set"

  Include {

    Options {

      signature = MD5

        compression = GZIP

    }

 

Not sure what you mean Compression=gzip in all of them.

 

Thank you,

Ankush More

 

From: Heitor Faria [mailto:heitor AT bacula.com DOT br]
Sent: Wednesday, March 23, 2016 8:14 PM
To: More, Ankush
Cc: compdoc AT hotrodpc DOT com; bacula-users AT lists.sourceforge DOT net
Subject: Re: [Bacula-users] Client backup with compress

 

Hello, Ankush:   ZLIB support:            yes

Your client is fine. It must be a FileSet configuration thing, since you seems to have multiple Options sub-resources. Just put Compression=gzip in all of them.

 

Regards,

---

From: "More, Ankush" <ankush.more AT capgemini DOT com>
To: compdoc AT hotrodpc DOT com, bacula-users AT lists.sourceforge DOT net
Sent: Wednesday, March 23, 2016 11:32:34 AM
Subject: Re: [Bacula-users] Client backup with compress

Hello,

 

Please see below, this is build for bacula client.

 

Configuration on Wed Mar 23 04:37:28 EDT 2016:

 

   Host:                    x86_64-unknown-linux-gnu -- redhat Enterprise release

   Bacula version:          Bacula 7.0.4 (04 June 2014)

   Source code location:    .

   Install binaries:        /usr/sbin

   Install libraries:       /usr/lib64

   Install config files:    /etc/bacula

   Scripts directory:       /usr/libexec/bacula/

   Archive directory:       /tmp

   Working directory:       /var/spool/bacula/

   PID directory:           /var/run

   Subsys directory:        /var/lock/subsys

   Man directory:           ${datarootdir}/man

   Data directory:          /usr/share

   Plugin directory:        /usr/lib64

   C Compiler:              gcc 4.1.2

   C++ Compiler:            /usr/bin/g++ 4.1.2

   Compiler flags:           -g -O2 -Wall -fno-strict-aliasing -fno-exceptions -fno-rtti

   Linker flags:

   Libraries:               -lpthread -ldl -ldl

   Statically Linked Tools: no

   Statically Linked FD:    no

   Statically Linked SD:    no

   Statically Linked DIR:   no

   Statically Linked CONS:  no

   Database backends:       None

   Database port:

   Database name:           bacula

   Database user:           bacula

 

   Job Output Email:        root@localhost

   Traceback Email:         root@localhost

   SMTP Host Address:       localhost

 

   Director Port:           9101

   File daemon Port:        9102

   Storage daemon Port:     9103

 

   Director User:

   Director Group:

   Storage Daemon User:

   Storage DaemonGroup:

   File Daemon User:

   File Daemon Group:

 

   Large file support:      yes

   Bacula conio support:    yes -lncurses

   readline support:        no

   TCP Wrappers support:    no

   TLS support:             yes

   Encryption support:      yes

   ZLIB support:            yes

   LZO support:             no

   enable-smartalloc:       yes

   enable-lockmgr:          no

   bat support:             no

   client-only:             yes

   build-dird:              no

   build-stored:            no

   Plugin support:          yes

   AFS support:             no

   ACL support:             yes

   XATTR support:           yes

   systemd support:         no

   Batch insert enabled:    None

 

Thank you,

Ankush More

 

From: compdoc [mailto:compdoc AT hotrodpc DOT com]
Sent: Wednesday, March 23, 2016 7:48 PM
To: bacula-users AT lists.sourceforge DOT net
Subject: Re: [Bacula-users] Client backup with compress

 

>#./configure --enable-client-only --enable-build-dird=no --enable-build-stored=no --bindir=/usr/bin --sbindir=/usr/sbin --with-scriptdir=/usr/libexec/bacula/ --with-working-dir=/var/spool/bacula/ --with-logdir=/var/log --enable-smartalloc

 

 

 

...

[Message clipped]  
------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

 

This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message.

------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785471&iu=/4140
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users