Bacula-users

Re: [Bacula-users] trouble inserting Attributes

2009-10-22 16:13:28
Subject: Re: [Bacula-users] trouble inserting Attributes
From: Christian Schnelle <cs AT igosys DOT de>
To: ebollengier <eric AT eb.homelinux DOT org>
Date: Thu, 22 Oct 2009 22:09:52 +0200
hi,

ebollengier schrieb:
> 
> 
> Christian Schnelle-2 wrote:
>> Hi,
>>
>> after an upgrade from bacula 1.38 to 2.2.4 i got trouble with one
>> backup job. This jobs backup round about 2TB data.
>> After writing the data to the tapes bconsole shows:
>> 'Dir inserting Attributes'.
>> This inserting runs now for about 3 days.
>>
>> Postgres shows:
>>
>> --snip--
>> template1=# select * from pg_stat_activity;
>>   datid |  datname  | procpid | usesysid | usename  |
>>
>>                                 current_query
>>
>>                          | waiting |          xact_start           |
>>        query_start          |         backend_start         |
>> client_addr | client_port
>> -------+-----------+---------+----------+----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------+-------------------------------+-------------------------------+-------------------------------+-------------+-------------
>>   16596 | bacula    |   26795 |    16384 | bacula   | <IDLE>
>>
>>
>>
>>                          | f       |                               |
>> 2009-10-22 11:04:53.904619+02 | 2009-10-17 22:10:00.599108+02 |
>>      |          -1
>>   16596 | bacula    |   20226 |    16384 | bacula   | INSERT INTO File
>> (FileIndex, JobId, PathId, FilenameId, LStat, MD5)SELECT
>> batch.FileIndex, batch.JobId, Path.PathId,
>> Filename.FilenameId,batch.LStat, batch.MD5 FROM batch JOIN Path ON
>> (batch.Path = Path.Path) JOIN Filename ON (batch.Name = Filename.Name) |
>> f       | 2009-10-21 06:08:49.528248+02 | 2009-10-21 06:08:49.528248+02
>> | 2009-10-19 03:44:09.545097+02 |             |          -1
>>       1 | template1 |   12282 |       10 | postgres | select * from
>> pg_stat_activity;
>>
>>
>>                             | f       | 2009-10-22 11:07:53.705321+02 |
>> 2009-10-22 11:07:53.705321+02 | 2009-10-22 11:07:52.013214+02 |
>>      |          -1
>> --snap--
>>
>> The backup machine is an dell poweredge r200 with dual xeon, 2gb ram,
>> 1gb /tmp and 230gb other disc space. it runs on debian lenny with cryted
>> filesystem.
>> There are no other servers than bacula running on this machine.
>> A Superloader 3 with 16 lto2 tapes works as tape storage.
>>
>> The postgresql config is this:
>>
>> --snip--
>> shared_buffers = 1024MB                 # min 128kB or
>> max_connections*16kB
>> temp_buffers = 160MB                    # min 800kB
>> max_prepared_transactions = 50          # can be 0 or more
>> work_mem = 92MB                         # min 64kB
>> maintenance_work_mem = 256MB            # min 1MB
>> max_stack_depth = 6MB                   # min 100kB
>> max_fsm_pages = 33000                   # min max_fsm_relations*16, 6
>> bytes each
>> max_fsm_relations = 2000                # min 100, ~70 bytes each
>> max_files_per_process = 1000
>> fsync = on                              # turns forced synchronization
>> on or off
>> synchronous_commit = on         # immediate fsync at commit
>> wal_sync_method = fsync         # the default is the first option
>> full_page_writes = on                   # recover from partial page writes
>> wal_buffers = 128kB                     # min 32kB
>>
>> wal_writer_delay = 200ms                # 1-10000 milliseconds
>> commit_delay = 0                        # range 0-100000, in microsec
>> commit_siblings = 5                     # range 1-1000
>> checkpoint_segments = 15                # in logfile segments 16MB each
>> checkpoint_timeout = 30s
>> --snap--
>>
>> Before upgrading to lenny it runs a debian sarge with bacula 1.38 and
>> anything worked like a charm. But since this upgrade this one backup job
>> lets me goning crazy.
>>
>> Please help me to fix this issue.
>>
>> Best regards
>>
>> Christian
>>
> 
> Hello Christian,
> 
> How many files are you pushing into the catalog for 2TB?

It's about 13 million files.

> You can start by
> changing the bad index
> on the File table like described in :
> http://sourceforge.net/apps/wordpress/bacula/2009/09/28/performance-issue-with-a-useless-index-on-postgresql/
> 
> Then, you can look at iostat and top to see if you are too short with CPU or
> disk usage (i think that
> your WAL/checkpoint settings can be enlarge, check your postgresql logs).

Ok, will do.

> The crypted filesystem should not help for the catalog.
> 
> Bye
> 

Thanks.

Cheers,

   Christian


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay 
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

<Prev in Thread] Current Thread [Next in Thread>