Bacula-users
[Top] [All Lists]
<prev [Date] next>
[Advanced]
 <prev [Thread] next>

Re: [Bacula-users] Strange issue with backup size

2013-04-06 09:11:23
Subject: Re: [Bacula-users] Strange issue with backup size
From: "Alberto Caporro" <a.caporro AT consulthink DOT it>
To: bacula-users AT lists.sourceforge DOT net
Date: Sat, 06 Apr 2013 15:07:12 +0200 
Hi everybody,

I have an update on this; last night the backup job ran again, this time 
as an incremental one. The job terminated successfully, but backup size 
again doesn't make sense, being roughly 88G (see log at the end of the 
mail).

A few files have changed for sure, being the machine a mail server, but 
nothing justifies such a huge size; just for comparison prior to 
installing Bacula backup was made with a custom script relying on 
duplicity, and incremental backup were sized at 2-3G.

Again this seems to have something to do with the 80G sparse file (and 
possibly a few other similar files) not recognised as this.

Regards,
Alberto

---

06-Apr 02:05 mnemosine.lab.retis.it-dir JobId 56: Start Backup JobId 56, 
Job=BackupMailserver.2013-04-06_02.05.00_20
06-Apr 02:05 mnemosine.lab.retis.it-dir JobId 56: Using Device 
"FileStorage"
06-Apr 02:05 mail.lab.retis.it-fd JobId 56: shell command: run 
ClientRunBeforeJob "/etc/init.d/zimbra stop"
06-Apr 02:05 mail.lab.retis.it-fd JobId 56: ClientRunBeforeJob: Host 
mail.mseclab.com
06-Apr 02:05 mail.lab.retis.it-fd JobId 56: ClientRunBeforeJob: 
        Stopping vmware-ha...Done.
06-Apr 02:05 mail.lab.retis.it-fd JobId 56: ClientRunBeforeJob: 
        Stopping zmconfigd...Done.
06-Apr 02:05 mail.lab.retis.it-fd JobId 56: ClientRunBeforeJob: 
        Stopping stats...Done.
06-Apr 02:05 mail.lab.retis.it-fd JobId 56: ClientRunBeforeJob: 
        Stopping mta...Done.
06-Apr 02:05 mail.lab.retis.it-fd JobId 56: ClientRunBeforeJob: 
        Stopping spell...Done.
06-Apr 02:05 mail.lab.retis.it-fd JobId 56: ClientRunBeforeJob: 
        Stopping snmp...Done.
06-Apr 02:05 mail.lab.retis.it-fd JobId 56: ClientRunBeforeJob: 
        Stopping cbpolicyd...Done.
06-Apr 02:05 mail.lab.retis.it-fd JobId 56: ClientRunBeforeJob: 
        Stopping archiving...Done.
06-Apr 02:05 mail.lab.retis.it-fd JobId 56: ClientRunBeforeJob: 
        Stopping antivirus...Done.
06-Apr 02:05 mail.lab.retis.it-fd JobId 56: ClientRunBeforeJob: 
        Stopping antispam...Done.
06-Apr 02:05 mail.lab.retis.it-fd JobId 56: ClientRunBeforeJob: 
        Stopping proxy...Done.
06-Apr 02:05 mail.lab.retis.it-fd JobId 56: ClientRunBeforeJob: 
        Stopping memcached...Done.
06-Apr 02:05 mail.lab.retis.it-fd JobId 56: ClientRunBeforeJob: 
        Stopping mailbox...Done.
06-Apr 02:05 mail.lab.retis.it-fd JobId 56: ClientRunBeforeJob: 
        Stopping logger...Done.
06-Apr 02:06 mail.lab.retis.it-fd JobId 56: ClientRunBeforeJob: 
        Stopping ldap...Done.
06-Apr 02:06 mnemosine.lab.retis.it-dir JobId 56: Sending Accurate 
information.
06-Apr 02:06 mnemosine.lab.retis.it-sd JobId 56: Volume "Volume011" 
previously written, moving to end of data.
06-Apr 02:06 mnemosine.lab.retis.it-sd JobId 56: Ready to append to end 
of Volume "Volume011" size=37788889773
06-Apr 02:37 mnemosine.lab.retis.it-sd JobId 56: End of medium on Volume 
"Volume011" Bytes=53,687,033,311 Blocks=832,208 at 06-Apr-2013 02:37.
06-Apr 02:38 mnemosine.lab.retis.it-sd JobId 56: Wrote label to 
prelabeled Volume "Volume012" on device "FileStorage" (/backup/data)
06-Apr 02:38 mnemosine.lab.retis.it-sd JobId 56: New volume "Volume012" 
mounted on device "FileStorage" (/backup/data) at 06-Apr-2013 02:38.
06-Apr 04:12 mnemosine.lab.retis.it-sd JobId 56: User defined maximum 
volume capacity 53,687,091,200 exceeded on device "FileStorage" 
(/backup/data).
06-Apr 04:12 mnemosine.lab.retis.it-sd JobId 56: End of medium on Volume 
"Volume012" Bytes=53,687,079,632 Blocks=832,203 at 06-Apr-2013 04:12.
06-Apr 04:13 mnemosine.lab.retis.it-sd JobId 56: Wrote label to 
prelabeled Volume "Volume013" on device "FileStorage" (/backup/data)
06-Apr 04:13 mnemosine.lab.retis.it-sd JobId 56: New volume "Volume013" 
mounted on device "FileStorage" (/backup/data) at 06-Apr-2013 04:13.
06-Apr 04:45 mail.lab.retis.it-fd JobId 56: shell command: run 
ClientAfterJob "/etc/init.d/zimbra start"
06-Apr 04:45 mnemosine.lab.retis.it-sd JobId 56: Job write elapsed time 
= 02:36:47, Transfer rate = 9.358 M Bytes/second
06-Apr 04:45 mail.lab.retis.it-fd JobId 56: ClientAfterJob: Host 
mail.mseclab.com
06-Apr 04:45 mail.lab.retis.it-fd JobId 56: ClientAfterJob:     Starting 
ldap...Done.
06-Apr 04:46 mail.lab.retis.it-fd JobId 56: ClientAfterJob:     Starting 
zmconfigd...Done.
06-Apr 04:46 mail.lab.retis.it-fd JobId 56: ClientAfterJob:     Starting 
logger...Done.
06-Apr 04:47 mail.lab.retis.it-fd JobId 56: ClientAfterJob:     Starting 
mailbox...Done.
06-Apr 04:47 mail.lab.retis.it-fd JobId 56: ClientAfterJob:     Starting 
antispam...Done.
06-Apr 04:47 mail.lab.retis.it-fd JobId 56: ClientAfterJob:     Starting 
antivirus...Done.
06-Apr 04:47 mail.lab.retis.it-fd JobId 56: ClientAfterJob:     Starting 
snmp...Done.
06-Apr 04:47 mail.lab.retis.it-fd JobId 56: ClientAfterJob:     Starting 
spell...Done.
06-Apr 04:47 mail.lab.retis.it-fd JobId 56: ClientAfterJob:     Starting 
mta...Done.
06-Apr 04:47 mail.lab.retis.it-fd JobId 56: ClientAfterJob:     Starting 
stats...Done.
06-Apr 04:47 mnemosine.lab.retis.it-dir JobId 56: Bacula 
mnemosine.lab.retis.it-dir 5.2.5 (26Jan12):
   Build OS:               x86_64-pc-linux-gnu ubuntu 12.04
   JobId:                  56
   Job:                    BackupMailserver.2013-04-06_02.05.00_20
   Backup Level:           Incremental, since=2013-04-05 02:05:51
   Client:                 "mail.lab.retis.it-fd" 5.0.1 (24Feb10) 
x86_64-pc-linux-gnu,ubuntu,10.04
   FileSet:                "FileServer" 2013-04-03 10:02:41
   Pool:                   "File" (From Job resource)
   Catalog:                "MyCatalog" (From Client resource)
   Storage:                "File" (From Job resource)
   Scheduled time:         06-Apr-2013 02:05:00
   Start time:             06-Apr-2013 02:06:01
   End time:               06-Apr-2013 04:47:48
   Elapsed time:           2 hours 41 mins 47 secs
   Priority:               20
   FD Files Written:       9,348
   SD Files Written:       9,348
   FD Bytes Written:       88,036,074,816 (88.03 GB)
   SD Bytes Written:       88,038,302,349 (88.03 GB)
   Rate:                   9069.3 KB/s
   Software Compression:   None
   VSS:                    no
   Encryption:             yes
   Accurate:               yes
   Volume name(s):         Volume011|Volume012|Volume013
   Volume Session Id:      42
   Volume Session Time:    1364943276
   Last Volume Bytes:      18,518,643,217 (18.51 GB)
   Non-fatal FD errors:    0
   SD Errors:              0
   FD termination status:  OK
   SD termination status:  OK
   Termination:            Backup OK

06-Apr 04:47 mnemosine.lab.retis.it-dir JobId 56: Begin pruning Jobs 
older than 6 months .
06-Apr 04:47 mnemosine.lab.retis.it-dir JobId 56: No Jobs found to 
prune.
06-Apr 04:47 mnemosine.lab.retis.it-dir JobId 56: Begin pruning Files.
06-Apr 04:47 mnemosine.lab.retis.it-dir JobId 56: No Files found to 
prune.
06-Apr 04:47 mnemosine.lab.retis.it-dir JobId 56: End auto prune.


---
Alberto Caporro
Director of Security Services - Consulthink Srl
Via dell'Archivio di Stato, 15
00144 Rome, Italy
http://www.consulthink.it
M. +39 335 1217 394
F. +39 06 4549 2454

GPG key fingerprint
46B5 2D38 2D48 3EFA 5C12 B86B 157D 80BA 2EAD DF41
On 5 Apr 2013, at 19:43, Alberto Caporro wrote:

> Hi Adrian,
>
> the "estimate" command gives more or less what I actually get:
>
> *estimate job=BackupMailserver
> Using Catalog "MyCatalog"
> Connecting to Client mail.lab.retis.it-fd at 10.10.200.10:9102
> 2000 OK estimate files=417,626 bytes=145,337,578,064
>
> It seems that despite the "sparse = yes" option the big file is being 
> counted in as with its "apparent" size.
>
> Just to clarify, this is what I get for the apparent and real size of 
> that file
>
> root@mail:/opt/zimbra/data/ldap/mdb/db# ls -l data.mdb && du -hs 
> data.mdb
> -rw------- 1 zimbra zimbra 85899345920 Apr  5 17:18 data.mdb
> 1.9M  data.mdb
>
> Subtracting those 80G from 145G I get 65G, which is a much closer 
> approximation to the real size of the fileset to be backed up.
>
> Regards,
> Alberto
>
>
> ---
> Alberto Caporro
> Director of Security Services - Consulthink Srl
> Via dell'Archivio di Stato, 15
> 00144 Rome, Italy
> http://www.consulthink.it
> M. +39 335 1217 394
> F. +39 06 4549 2454
>
> GPG key fingerprint
> 46B5 2D38 2D48 3EFA 5C12 B86B 157D 80BA 2EAD DF41
> On 5 Apr 2013, at 18:59, Adrian Reyer wrote:
>
>> Hi Alberto,
>>
>> On Fri, Apr 05, 2013 at 12:58:22PM +0200, Alberto Caporro wrote:
>>> Hi Adrian, thanks for your advice, I was actually already thinking 
>>> of
>>> switching to a full backup strategy; unfortunately that would not 
>>> solve
>>> tho size issue, which at this point is becoming quite puzzling :-)
>>
>> Sorry, I read your original issue already as solved.
>> Perhaps the 'estimate' command is helpful:
>> estimate job=YOURJOB level=Full listing
>> should give you a file list of what is actually backed up. Perhaps 
>> this
>> sheds some light.
>> Other posiblities depend on your mailserver software/storage format 
>> and
>> filesystem.  How are the sizes compared with
>> - df
>> - du -s dir
>> - tar cf - dir | wc
>> - tar cSf - dir | wc
>>
>> Regards,
>>      Adrian
>> --
>> LiHAS - Adrian Reyer - Hessenwiesenstraße 10 - D-70565 Stuttgart
>> Fon: +49 (7 11) 78 28 50 90 - Fax:  +49 (7 11) 78 28 50 91
>> Mail: lihas AT lihas DOT de - Web: http://lihas.de
>> Linux, Netzwerke, Consulting & Support - USt-ID: DE 227 816 626 
>> Stuttgart

------------------------------------------------------------------------------
Minimize network downtime and maximize team effectiveness.
Reduce network management and security costs.Learn how to hire 
the most talented Cisco Certified professionals. Visit the 
Employer Resources Portal
http://www.cisco.com/web/learning/employer_resources/index.html
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
[More with this subject...]
<Prev in Thread] Current Thread [Next in Thread>
Previous by Date:  [Bacula-users] Trying unsuccesfully to implement Vchanger for encrypted off site backups., Mike Lindsay
Next by Date:  Re: [Bacula-users] Strange issue with backup size, ganiuszka
Previous by Thread:  Re: [Bacula-users] Strange issue with backup size, Alberto Caporro
Next by Thread:  Re: [Bacula-users] Strange issue with backup size, ganiuszka
Indexes:  [Date] [Thread] [Top] [All Lists]