Bacula-users
[Top] [All Lists]
<prev [Date] next>
[Advanced]
 <prev [Thread] next>

Re: [Bacula-users] Improving Backup speed

2008-11-18 08:42:56
Subject: Re: [Bacula-users] Improving Backup speed
From: Pascal Clermont <pascal-list AT LEXUM.UMontreal DOT CA>
To: Dan Langille <dan AT langille DOT org>
Date: Tue, 18 Nov 2008 08:40:45 -0500 
Dan Langille wrote:
> On Nov 17, 2008, at 4:03 PM, Pascal Clermont wrote:
> 
>> Hi,
>> Currently spending a lot of time on improving the speeds of this  
>> network.
>> After several tests I have realized that the database is "inserting
>> attributes" was taking quite a lot of time. In order to improve this I
>> have searched your forums and see that using batch mode would increase
>> this action by 10 times. The guide states:
>> " One way is to see if the PostgreSQL library that Bacula will be  
>> linked
>> against references pthreads. This can be done with a command such as:
>>   nm /usr/lib/libpq.a | grep pthread_mutex_lock ".
>>
>> The file /usr/lib/libpq.a does not exist, I am using postgreSQL 8.3.5
>> and did not install from source, but from the package manager that  
>> comes
>> with the OS, YUM.
> 
> The problem with operating systems is they each think theirs is the  
> file layout.
> 
> Experiment a little, and you'll find it.  Mine, on FreeBSD, is here:
> 
> $  nm /usr/local/lib/libpq.a | grep pthread_mutex_lock
>           U pthread_mutex_lock
>           U pthread_mutex_lock
> 
> 
>>
>> Would someone know how I can find out if thread safety is on without
>> breaking everything?
>> AND if it is on, will Batch Insert Code from bacula work anyways?  
>> since
>> the file it links against "/usr/lib/libpq.a" is not existent.
> 
> I'm sure you have libpq.a
> 
> Just keep in mind that location varies.
> 
>>
>>
>> Pascal S. Clermont
>>
>> -------------------------------------------------------------------------
>> This SF.Net email is sponsored by the Moblin Your Move Developer's  
>> challenge
>> Build the coolest Linux based applications with Moblin SDK & win  
>> great prizes
>> Grand prize is a trip for two to an Open Source event anywhere in  
>> the world
>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>> _______________________________________________
>> Bacula-users mailing list
>> Bacula-users AT lists.sourceforge DOT net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
> 
Ok here is the steps I took that made me believe that I do not have libpq.a:

[root@vicenza bacula]# find / -iname libpq.a
[root@vicenza bacula]#


[root@vicenza bacula]# updatedb && locate libpq.a
[root@vicenza bacula]#


[root@vicenza bacula]# whereis libpq.a
libpq:


[root@vicenza bacula]# rpm -ql postgresql-server
/etc/pam.d/postgresql
/etc/rc.d/init.d/postgresql
/etc/sysconfig/pgsql
/usr/bin/initdb
/usr/bin/ipcclean
/usr/bin/pg_controldata
/usr/bin/pg_ctl
/usr/bin/pg_resetxlog
/usr/bin/postgres
/usr/bin/postmaster
/usr/lib/pgsql
/usr/lib/pgsql/ascii_and_mic.so
/usr/lib/pgsql/cyrillic_and_mic.so
/usr/lib/pgsql/dict_int.so
/usr/lib/pgsql/dict_snowball.so
/usr/lib/pgsql/dict_xsyn.so
/usr/lib/pgsql/euc_cn_and_mic.so
/usr/lib/pgsql/euc_jis_2004_and_shift_jis_2004.so
/usr/lib/pgsql/euc_jp_and_sjis.so
/usr/lib/pgsql/euc_kr_and_mic.so
/usr/lib/pgsql/euc_tw_and_big5.so
/usr/lib/pgsql/latin2_and_win1250.so
/usr/lib/pgsql/latin_and_mic.so
/usr/lib/pgsql/plpgsql.so
/usr/lib/pgsql/test_parser.so
/usr/lib/pgsql/tsearch2.so
/usr/lib/pgsql/utf8_and_ascii.so
/usr/lib/pgsql/utf8_and_big5.so
/usr/lib/pgsql/utf8_and_cyrillic.so
/usr/lib/pgsql/utf8_and_euc_cn.so
/usr/lib/pgsql/utf8_and_euc_jis_2004.so
/usr/lib/pgsql/utf8_and_euc_jp.so
/usr/lib/pgsql/utf8_and_euc_kr.so
/usr/lib/pgsql/utf8_and_euc_tw.so
/usr/lib/pgsql/utf8_and_gb18030.so
/usr/lib/pgsql/utf8_and_gbk.so
/usr/lib/pgsql/utf8_and_iso8859.so
/usr/lib/pgsql/utf8_and_iso8859_1.so
/usr/lib/pgsql/utf8_and_johab.so
/usr/lib/pgsql/utf8_and_shift_jis_2004.so
/usr/lib/pgsql/utf8_and_sjis.so
/usr/lib/pgsql/utf8_and_uhc.so
/usr/lib/pgsql/utf8_and_win.so
/usr/share/locale/af/LC_MESSAGES/postgres.mo
/usr/share/locale/cs/LC_MESSAGES/pg_controldata.mo
/usr/share/locale/cs/LC_MESSAGES/pg_resetxlog.mo
/usr/share/locale/cs/LC_MESSAGES/postgres.mo
/usr/share/locale/de/LC_MESSAGES/pg_controldata.mo
/usr/share/locale/de/LC_MESSAGES/pg_resetxlog.mo
/usr/share/locale/de/LC_MESSAGES/postgres.mo
/usr/share/locale/es/LC_MESSAGES/pg_controldata.mo
/usr/share/locale/es/LC_MESSAGES/pg_resetxlog.mo
/usr/share/locale/es/LC_MESSAGES/postgres.mo
/usr/share/locale/fa/LC_MESSAGES/pg_controldata.mo
/usr/share/locale/fr/LC_MESSAGES/pg_controldata.mo
/usr/share/locale/fr/LC_MESSAGES/pg_resetxlog.mo
/usr/share/locale/fr/LC_MESSAGES/postgres.mo
/usr/share/locale/hr/LC_MESSAGES/postgres.mo
/usr/share/locale/hu/LC_MESSAGES/pg_controldata.mo
/usr/share/locale/hu/LC_MESSAGES/pg_resetxlog.mo
/usr/share/locale/hu/LC_MESSAGES/postgres.mo
/usr/share/locale/it/LC_MESSAGES/pg_controldata.mo
/usr/share/locale/it/LC_MESSAGES/pg_resetxlog.mo
/usr/share/locale/it/LC_MESSAGES/postgres.mo
/usr/share/locale/ko/LC_MESSAGES/pg_controldata.mo
/usr/share/locale/ko/LC_MESSAGES/pg_resetxlog.mo
/usr/share/locale/ko/LC_MESSAGES/postgres.mo
/usr/share/locale/nb/LC_MESSAGES/pg_controldata.mo
/usr/share/locale/nb/LC_MESSAGES/pg_resetxlog.mo
/usr/share/locale/nb/LC_MESSAGES/postgres.mo
/usr/share/locale/nl/LC_MESSAGES/postgres.mo
/usr/share/locale/pl/LC_MESSAGES/pg_controldata.mo
/usr/share/locale/pl/LC_MESSAGES/postgres.mo
/usr/share/locale/pt_BR/LC_MESSAGES/pg_controldata.mo
/usr/share/locale/pt_BR/LC_MESSAGES/pg_resetxlog.mo
/usr/share/locale/pt_BR/LC_MESSAGES/postgres.mo
/usr/share/locale/ro/LC_MESSAGES/pg_controldata.mo
/usr/share/locale/ro/LC_MESSAGES/pg_resetxlog.mo
/usr/share/locale/ro/LC_MESSAGES/postgres.mo
/usr/share/locale/ru/LC_MESSAGES/pg_controldata.mo
/usr/share/locale/ru/LC_MESSAGES/pg_resetxlog.mo
/usr/share/locale/ru/LC_MESSAGES/postgres.mo
/usr/share/locale/sk/LC_MESSAGES/pg_controldata.mo
/usr/share/locale/sk/LC_MESSAGES/pg_resetxlog.mo
/usr/share/locale/sk/LC_MESSAGES/postgres.mo
/usr/share/locale/sl/LC_MESSAGES/pg_controldata.mo
/usr/share/locale/sl/LC_MESSAGES/pg_resetxlog.mo
/usr/share/locale/sl/LC_MESSAGES/postgres.mo
/usr/share/locale/sv/LC_MESSAGES/pg_controldata.mo
/usr/share/locale/sv/LC_MESSAGES/pg_resetxlog.mo
/usr/share/locale/sv/LC_MESSAGES/postgres.mo
/usr/share/locale/ta/LC_MESSAGES/pg_controldata.mo
/usr/share/locale/ta/LC_MESSAGES/pg_resetxlog.mo
/usr/share/locale/tr/LC_MESSAGES/pg_controldata.mo
/usr/share/locale/tr/LC_MESSAGES/pg_resetxlog.mo
/usr/share/locale/tr/LC_MESSAGES/postgres.mo
/usr/share/locale/zh_CN/LC_MESSAGES/pg_controldata.mo
/usr/share/locale/zh_CN/LC_MESSAGES/pg_resetxlog.mo
/usr/share/locale/zh_CN/LC_MESSAGES/postgres.mo
/usr/share/locale/zh_TW/LC_MESSAGES/pg_controldata.mo
/usr/share/locale/zh_TW/LC_MESSAGES/pg_resetxlog.mo
/usr/share/locale/zh_TW/LC_MESSAGES/postgres.mo
/usr/share/man/man1/initdb.1.gz
/usr/share/man/man1/ipcclean.1.gz
/usr/share/man/man1/pg_controldata.1.gz
/usr/share/man/man1/pg_ctl.1.gz
/usr/share/man/man1/pg_resetxlog.1.gz
/usr/share/man/man1/postgres.1.gz
/usr/share/man/man1/postmaster.1.gz
/usr/share/pgsql
/usr/share/pgsql/conversion_create.sql
/usr/share/pgsql/information_schema.sql
/usr/share/pgsql/pg_hba.conf.sample
/usr/share/pgsql/pg_ident.conf.sample
/usr/share/pgsql/pg_service.conf.sample
/usr/share/pgsql/postgres.bki
/usr/share/pgsql/postgres.description
/usr/share/pgsql/postgres.shdescription
/usr/share/pgsql/postgresql.conf.sample
/usr/share/pgsql/psqlrc.sample
/usr/share/pgsql/recovery.conf.sample
/usr/share/pgsql/snowball_create.sql
/usr/share/pgsql/sql_features.txt
/usr/share/pgsql/system_views.sql
/usr/share/pgsql/timezonesets/Africa.txt
/usr/share/pgsql/timezonesets/America.txt
/usr/share/pgsql/timezonesets/Antarctica.txt
/usr/share/pgsql/timezonesets/Asia.txt
/usr/share/pgsql/timezonesets/Atlantic.txt
/usr/share/pgsql/timezonesets/Australia
/usr/share/pgsql/timezonesets/Australia.txt
/usr/share/pgsql/timezonesets/Default
/usr/share/pgsql/timezonesets/Etc.txt
/usr/share/pgsql/timezonesets/Europe.txt
/usr/share/pgsql/timezonesets/India
/usr/share/pgsql/timezonesets/Indian.txt
/usr/share/pgsql/timezonesets/Pacific.txt
/usr/share/pgsql/tsearch_data/danish.stop
/usr/share/pgsql/tsearch_data/dutch.stop
/usr/share/pgsql/tsearch_data/english.stop
/usr/share/pgsql/tsearch_data/finnish.stop
/usr/share/pgsql/tsearch_data/french.stop
/usr/share/pgsql/tsearch_data/german.stop
/usr/share/pgsql/tsearch_data/hungarian.stop
/usr/share/pgsql/tsearch_data/hunspell_sample.affix
/usr/share/pgsql/tsearch_data/ispell_sample.affix
/usr/share/pgsql/tsearch_data/ispell_sample.dict
/usr/share/pgsql/tsearch_data/italian.stop
/usr/share/pgsql/tsearch_data/norwegian.stop
/usr/share/pgsql/tsearch_data/portuguese.stop
/usr/share/pgsql/tsearch_data/russian.stop
/usr/share/pgsql/tsearch_data/spanish.stop
/usr/share/pgsql/tsearch_data/swedish.stop
/usr/share/pgsql/tsearch_data/synonym_sample.syn
/usr/share/pgsql/tsearch_data/thesaurus_sample.ths
/usr/share/pgsql/tsearch_data/turkish.stop
/usr/share/pgsql/tsearch_data/xsyn_sample.rules
/var/lib/pgsql
/var/lib/pgsql/.bash_profile
/var/lib/pgsql/backups
/var/lib/pgsql/data
[root@vicenza bacula]#

[root@vicenza bacula]# rpm -ql postgresql-libs
/usr/lib/libecpg.so.6
/usr/lib/libecpg.so.6.0
/usr/lib/libecpg_compat.so.3
/usr/lib/libecpg_compat.so.3.0
/usr/lib/libpgtypes.so.3
/usr/lib/libpgtypes.so.3.0
/usr/lib/libpq.so.5
/usr/lib/libpq.so.5.1
/usr/share/locale/af/LC_MESSAGES/libpq.mo
/usr/share/locale/cs/LC_MESSAGES/libpq.mo
/usr/share/locale/de/LC_MESSAGES/libpq.mo
/usr/share/locale/es/LC_MESSAGES/libpq.mo
/usr/share/locale/fr/LC_MESSAGES/libpq.mo
/usr/share/locale/hr/LC_MESSAGES/libpq.mo
/usr/share/locale/it/LC_MESSAGES/libpq.mo
/usr/share/locale/ko/LC_MESSAGES/libpq.mo
/usr/share/locale/nb/LC_MESSAGES/libpq.mo
/usr/share/locale/pl/LC_MESSAGES/libpq.mo
/usr/share/locale/pt_BR/LC_MESSAGES/libpq.mo
/usr/share/locale/ru/LC_MESSAGES/libpq.mo
/usr/share/locale/sk/LC_MESSAGES/libpq.mo
/usr/share/locale/sl/LC_MESSAGES/libpq.mo
/usr/share/locale/sv/LC_MESSAGES/libpq.mo
/usr/share/locale/ta/LC_MESSAGES/libpq.mo
/usr/share/locale/tr/LC_MESSAGES/libpq.mo
/usr/share/locale/zh_CN/LC_MESSAGES/libpq.mo
/usr/share/locale/zh_TW/LC_MESSAGES/libpq.mo
[root@vicenza bacula]#


####################### Some information on the OS and postgresql server

[root@vicenza bacula]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.2 (Tikanga)


[root@vicenza bacula]# rpm -qi postgresql-server
Name        : postgresql-server            Relocations: (not relocatable)
Version     : 8.3.5                             Vendor: (none)
Release     : 1PGDG.rhel5                   Build Date: Fri 31 Oct 2008 
02:04:30 PM EDT
Install Date: Mon 17 Nov 2008 12:01:23 PM EST      Build Host: 
koji-centos5-i386-pg83
Group       : Applications/Databases        Source RPM: 
postgresql-8.3.5-1PGDG.rhel5.src.rpm
Size        : 12406119                         License: BSD
Signature   : DSA/SHA1, Fri 31 Oct 2008 02:04:48 PM EDT, Key ID 
1f16d2e1442df0f8
URL         : http://www.postgresql.org/
Summary     : The programs needed to create and run a PostgreSQL server.
Description :
The postgresql-server package includes the programs needed to create
and run a PostgreSQL server, which will in turn allow you to create
and maintain PostgreSQL databases.  PostgreSQL is an advanced
Object-Relational database management system (DBMS) that supports
almost all SQL constructs (including transactions, subselects and
user-defined types and functions). You should install
postgresql-server if you want to create and maintain your own
PostgreSQL databases and/or your own PostgreSQL server. You also need
to install the postgresql package.



[root@vicenza bacula]# rpm -qi postgresql-libs
Name        : postgresql-libs              Relocations: (not relocatable)
Version     : 8.3.5                             Vendor: (none)
Release     : 1PGDG.rhel5                   Build Date: Fri 31 Oct 2008 
02:04:30 PM EDT
Install Date: Mon 17 Nov 2008 12:00:07 PM EST      Build Host: 
koji-centos5-i386-pg83
Group       : Applications/Databases        Source RPM: 
postgresql-8.3.5-1PGDG.rhel5.src.rpm
Size        : 556358                           License: BSD
Signature   : DSA/SHA1, Fri 31 Oct 2008 02:04:36 PM EDT, Key ID 
1f16d2e1442df0f8
URL         : http://www.postgresql.org/
Summary     : The shared libraries required for any PostgreSQL clients.
Description :
The postgresql-libs package provides the essential shared libraries for any
PostgreSQL client program or interface. You will need to install this 
package
to use any other PostgreSQL package or any clients that need to connect to a
PostgreSQL server.


I will keep hunting this down, any comments/tips is really appreciated :)

Pascal S. Clermont

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
[More with this subject...]
<Prev in Thread] Current Thread [Next in Thread>
Previous by Date:  [Bacula-users] syntax error for "File" entry?, Win Htin
Next by Date:  [Bacula-users] Bacula Catalog File - How to reduce its size, Rodrigo Rosenfeld Rosas
Previous by Thread:  Re: [Bacula-users] Improving Backup speed, Dan Langille
Next by Thread:  Re: [Bacula-users] Improving Backup speed, Frank Sweetser
Indexes:  [Date] [Thread] [Top] [All Lists]