Bacula-users

Re: [Bacula-users] very slow backups, around 5mb/sec

2016-06-21 22:33:58
Subject: Re: [Bacula-users] very slow backups, around 5mb/sec
From: Wanderlei Huttel <wanderleihuttel AT gmail DOT com>
To: bacula-users AT lists.sourceforge DOT net
Date: Tue, 21 Jun 2016 23:32:54 -0300

Hello, have you tried to change the bacula database for MySQL or Postgres?

Best regards

Wanderlei Hüttel
Enviado de Motorola Moto X2

Em 21 de jun de 2016 11:21 PM, "fgd9329g" <bacula-forum AT backupcentral DOT com> escreveu:
This might be a dumb question, but I can't figure it out.

Debian 8
Bacula Version: 5.2.6
sqlite3 -version 3.8.7.1
Backing up to disks, not tape.

I have two bacula systems, both have this same slowness issue. Here's what the final output of our largest job looks like:

Elapsed time:           4 days 1 hour 58 mins 23 secs
Priority:               10
FD Files Written:       20,103,048
SD Files Written:       20,103,048
FD Bytes Written:       922,921,450,734 (922.9 GB)
SD Bytes Written:       925,825,875,288 (925.8 GB)
Rate:                   2616.7 KB/s
Software Compression:   54.4 %
VSS:                    no
Encryption:             no
Accurate:               no

The bacula server and remote server both have 10GB network cards. On the bacula server, if I scp a directory from the remote server it will transfer at about 50MB/sec and a single multi-GB file will transfer close to 300MB/sec. So the network seems okay. Even an rsync takes ~40sec when bacula takes 12 minutes on the same directory.

The bacula and remote servers are pretty hefty too: 20+ cores, 32GB of ram, SAS hard drives, 10GB network cards. During the backup process they don't appear to be busy; one core is at 11%, the memory sits around 1gb used, iotop reports very low IO%.

I thought it might be a network issue, so I copied a directory locally to the bacula server then backed it up. It only went marginally faster, and here's that output:

Elapsed time:           12 mins 43 secs
Priority:               10
FD Files Written:       93,592
SD Files Written:       93,592
FD Bytes Written:       3,989,758,931 (3.989 GB)
SD Bytes Written:       4,000,333,487 (4.000 GB)
Rate:                   5229.0 KB/s
Software Compression:   67.9 %
VSS:                    no
Encryption:             no
Accurate:               no


The bacula-dir section for the big job looks like:
FileSet {
        Name = "mail set"
        Include {
                Options {
                        compression = GZIP
                        signature = MD5
                        }
                File = /usr
                File = /etc
                File = /var/spool/cyrus/mail
                File = /var
                }
        Exclude {
                File = /proc
                File = /tmp
                File = /.journal
                File = /.fsck
                File = /var/cache
                }
        }
Job {
    Name = "mail"
    JobDefs = "Default"
    Client = mail-fd
    FileSet = "mail set"
    Spool Attributes = yes
}


Turning off compression didn't seem to make much of a difference either. I just don't know what to do at this point. Does anybody have an idea for why these transfers are so slow through bacula?

+----------------------------------------------------------------------
|This was sent by c4555622 AT trbvn DOT com via Backup Central.
|Forward SPAM to abuse AT backupcentral DOT com.
+----------------------------------------------------------------------



------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
<Prev in Thread] Current Thread [Next in Thread>