Re: [Bacula-users] bextract consumes 3+Gb memory

Andy Howell wrote:
> Hello,
> 
>       I was testing extracting the catalog from disk volume, only to find the 
> machine swapping, 
> and bextract using more the 3Gb res mem, 4+Gb virtual. I had compression 
> turned on in the 
> catalog fileset. When I turned that off, ran BackupCatalog again to a new 
> disk volume, I 
> was able to extract the bacula.sql file.
> 
> I tried it on another disk voluem that was compressed and had the same 
> problem.
> 
> This is bacula 5.0.0 on CentOS 5.4.
> 
> Any ideas?

Sorry to reply to my own post. I found a bug in bextract. The length of the 
buffer for 
un-compressing records was not set. This results in the buffer requesting a 
huge chunk of 
memory. This patch fixes that:

diff -c bextract.c.org bextract.c
*** bextract.c.org      2010-02-11 18:40:12.000000000 -0600
--- bextract.c  2010-02-11 19:36:00.000000000 -0600
***************
*** 408,414 ****
      case STREAM_WIN32_GZIP_DATA:
   #ifdef HAVE_LIBZ
         if (extract) {
!          uLong compress_len;
            int stat;

            if (rec->Stream == STREAM_SPARSE_GZIP_DATA) {
--- 408,414 ----
      case STREAM_WIN32_GZIP_DATA:
   #ifdef HAVE_LIBZ
         if (extract) {
!          uLong compress_len = sizeof_pool_memory(compress_buf);
            int stat;

            if (rec->Stream == STREAM_SPARSE_GZIP_DATA) {

I'm still left with another problem. The extracted file is not the right size. 
There are 
two copies of the bacula.sql file in the disk volume. When bextract runs, there 
is no way 
to tell it which one. The extracted file ends up an order of magnitude smaller 
than it 
should be.

  "/strongbox/bacula/bacula" for reading.
11-Feb 19:42 bextract JobId 0: Ready to read from volume "bacula-0083" on 
device 
"baculaStorage" (/strongbox/bacula/bacula).
bextract JobId 0: -rw-------   1 bacula   bacula     497885461 2010-02-11 
00:01:42 
/tmp/var/lib/bacula/bacula.sql
bextract JobId 0: -rw-------   1 bacula   bacula     497816208 2010-02-11 
00:23:25 
/tmp/var/lib/bacula/bacula.sql
11-Feb 19:43 bextract JobId 0: End of Volume at file 0 on device 
"baculaStorage" 
(/strongbox/bacula/bacula), Volume "bacula-0083"
11-Feb 19:43 bextract JobId 0: End of all volumes.
11-Feb 19:43 bextract JobId 0: Error: attribs.c:423 File size of restored file 
/tmp/var/lib/bacula/bacula.sql not correct. Original 497816208, restored 
77201408.
2 files restored.

Not sure what is happnening here.

Regards,

        Andy


------------------------------------------------------------------------------
SOLARIS 10 is the OS for Data Centers - provides features such as DTrace,
Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW
http://p.sf.net/sfu/solaris-dev2dev
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users