Re: [Bacula-users] Improving Backup speed



--- On Tue, 11/18/08, Pascal Clermont <pascal-list AT LEXUM.UMontreal DOT CA> 
wrote:

> From: Pascal Clermont <pascal-list AT LEXUM.UMontreal DOT CA>
> Subject: Re: [Bacula-users] Improving Backup speed
> To: "Dan Langille" <dan AT langille DOT org>
> Cc: bacula-users AT lists.sourceforge DOT net
> Date: Tuesday, November 18, 2008, 7:40 AM
> Dan Langille wrote:
> > On Nov 17, 2008, at 4:03 PM, Pascal Clermont wrote:
> > 
> >> Hi,
> >> Currently spending a lot of time on improving the
> speeds of this  
> >> network.
> >> After several tests I have realized that the
> database is "inserting
> >> attributes" was taking quite a lot of time.
> In order to improve this I
> >> have searched your forums and see that using batch
> mode would increase
> >> this action by 10 times. The guide states:
> >> " One way is to see if the PostgreSQL library
> that Bacula will be  
> >> linked
> >> against references pthreads. This can be done with
> a command such as:
> >>   nm /usr/lib/libpq.a | grep pthread_mutex_lock
> ".
> >>
> >> The file /usr/lib/libpq.a does not exist, I am
> using postgreSQL 8.3.5
> >> and did not install from source, but from the
> package manager that  
> >> comes
> >> with the OS, YUM.
> > 
> > The problem with operating systems is they each think
> theirs is the  
> > file layout.
> > 
> > Experiment a little, and you'll find it.  Mine, on
> FreeBSD, is here:
> > 
> > $  nm /usr/local/lib/libpq.a | grep pthread_mutex_lock
> >           U pthread_mutex_lock
> >           U pthread_mutex_lock
> > 
> > 
> >>
> >> Would someone know how I can find out if thread
> safety is on without
> >> breaking everything?
> >> AND if it is on, will Batch Insert Code from
> bacula work anyways?  
> >> since
> >> the file it links against
> "/usr/lib/libpq.a" is not existent.
> > 
> > I'm sure you have libpq.a
> > 
> > Just keep in mind that location varies.
> > 
> >>
> >>
> >> Pascal S. Clermont
> >>
> >>
> -------------------------------------------------------------------------
> >> This SF.Net email is sponsored by the Moblin Your
> Move Developer's  
> >> challenge
> >> Build the coolest Linux based applications with
> Moblin SDK & win  
> >> great prizes
> >> Grand prize is a trip for two to an Open Source
> event anywhere in  
> >> the world
> >>
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> >> _______________________________________________
> >> Bacula-users mailing list
> >> Bacula-users AT lists.sourceforge DOT net
> >>
> https://lists.sourceforge.net/lists/listinfo/bacula-users
> > 
> Ok here is the steps I took that made me believe that I do
> not have libpq.a:
> 
> [root@vicenza bacula]# find / -iname libpq.a
> [root@vicenza bacula]#
> 
> 
> [root@vicenza bacula]# updatedb && locate libpq.a
> [root@vicenza bacula]#
> 
> 
> [root@vicenza bacula]# whereis libpq.a
> libpq:
> 
> 
> [root@vicenza bacula]# rpm -ql postgresql-server
> /etc/pam.d/postgresql
> /etc/rc.d/init.d/postgresql
> /etc/sysconfig/pgsql
> /usr/bin/initdb
> /usr/bin/ipcclean
> /usr/bin/pg_controldata
> /usr/bin/pg_ctl
> /usr/bin/pg_resetxlog
> /usr/bin/postgres
> /usr/bin/postmaster
> /usr/lib/pgsql
> /usr/lib/pgsql/ascii_and_mic.so
> /usr/lib/pgsql/cyrillic_and_mic.so
> /usr/lib/pgsql/dict_int.so
> /usr/lib/pgsql/dict_snowball.so
> /usr/lib/pgsql/dict_xsyn.so
> /usr/lib/pgsql/euc_cn_and_mic.so
> /usr/lib/pgsql/euc_jis_2004_and_shift_jis_2004.so
> /usr/lib/pgsql/euc_jp_and_sjis.so
> /usr/lib/pgsql/euc_kr_and_mic.so
> /usr/lib/pgsql/euc_tw_and_big5.so
> /usr/lib/pgsql/latin2_and_win1250.so
> /usr/lib/pgsql/latin_and_mic.so
> /usr/lib/pgsql/plpgsql.so
> /usr/lib/pgsql/test_parser.so
> /usr/lib/pgsql/tsearch2.so
> /usr/lib/pgsql/utf8_and_ascii.so
> /usr/lib/pgsql/utf8_and_big5.so
> /usr/lib/pgsql/utf8_and_cyrillic.so
> /usr/lib/pgsql/utf8_and_euc_cn.so
> /usr/lib/pgsql/utf8_and_euc_jis_2004.so
> /usr/lib/pgsql/utf8_and_euc_jp.so
> /usr/lib/pgsql/utf8_and_euc_kr.so
> /usr/lib/pgsql/utf8_and_euc_tw.so
> /usr/lib/pgsql/utf8_and_gb18030.so
> /usr/lib/pgsql/utf8_and_gbk.so
> /usr/lib/pgsql/utf8_and_iso8859.so
> /usr/lib/pgsql/utf8_and_iso8859_1.so
> /usr/lib/pgsql/utf8_and_johab.so
> /usr/lib/pgsql/utf8_and_shift_jis_2004.so
> /usr/lib/pgsql/utf8_and_sjis.so
> /usr/lib/pgsql/utf8_and_uhc.so
> /usr/lib/pgsql/utf8_and_win.so
> /usr/share/locale/af/LC_MESSAGES/postgres.mo
> /usr/share/locale/cs/LC_MESSAGES/pg_controldata.mo
> /usr/share/locale/cs/LC_MESSAGES/pg_resetxlog.mo
> /usr/share/locale/cs/LC_MESSAGES/postgres.mo
> /usr/share/locale/de/LC_MESSAGES/pg_controldata.mo
> /usr/share/locale/de/LC_MESSAGES/pg_resetxlog.mo
> /usr/share/locale/de/LC_MESSAGES/postgres.mo
> /usr/share/locale/es/LC_MESSAGES/pg_controldata.mo
> /usr/share/locale/es/LC_MESSAGES/pg_resetxlog.mo
> /usr/share/locale/es/LC_MESSAGES/postgres.mo
> /usr/share/locale/fa/LC_MESSAGES/pg_controldata.mo
> /usr/share/locale/fr/LC_MESSAGES/pg_controldata.mo
> /usr/share/locale/fr/LC_MESSAGES/pg_resetxlog.mo
> /usr/share/locale/fr/LC_MESSAGES/postgres.mo
> /usr/share/locale/hr/LC_MESSAGES/postgres.mo
> /usr/share/locale/hu/LC_MESSAGES/pg_controldata.mo
> /usr/share/locale/hu/LC_MESSAGES/pg_resetxlog.mo
> /usr/share/locale/hu/LC_MESSAGES/postgres.mo
> /usr/share/locale/it/LC_MESSAGES/pg_controldata.mo
> /usr/share/locale/it/LC_MESSAGES/pg_resetxlog.mo
> /usr/share/locale/it/LC_MESSAGES/postgres.mo
> /usr/share/locale/ko/LC_MESSAGES/pg_controldata.mo
> /usr/share/locale/ko/LC_MESSAGES/pg_resetxlog.mo
> /usr/share/locale/ko/LC_MESSAGES/postgres.mo
> /usr/share/locale/nb/LC_MESSAGES/pg_controldata.mo
> /usr/share/locale/nb/LC_MESSAGES/pg_resetxlog.mo
> /usr/share/locale/nb/LC_MESSAGES/postgres.mo
> /usr/share/locale/nl/LC_MESSAGES/postgres.mo
> /usr/share/locale/pl/LC_MESSAGES/pg_controldata.mo
> /usr/share/locale/pl/LC_MESSAGES/postgres.mo
> /usr/share/locale/pt_BR/LC_MESSAGES/pg_controldata.mo
> /usr/share/locale/pt_BR/LC_MESSAGES/pg_resetxlog.mo
> /usr/share/locale/pt_BR/LC_MESSAGES/postgres.mo
> /usr/share/locale/ro/LC_MESSAGES/pg_controldata.mo
> /usr/share/locale/ro/LC_MESSAGES/pg_resetxlog.mo
> /usr/share/locale/ro/LC_MESSAGES/postgres.mo
> /usr/share/locale/ru/LC_MESSAGES/pg_controldata.mo
> /usr/share/locale/ru/LC_MESSAGES/pg_resetxlog.mo
> /usr/share/locale/ru/LC_MESSAGES/postgres.mo
> /usr/share/locale/sk/LC_MESSAGES/pg_controldata.mo
> /usr/share/locale/sk/LC_MESSAGES/pg_resetxlog.mo
> /usr/share/locale/sk/LC_MESSAGES/postgres.mo
> /usr/share/locale/sl/LC_MESSAGES/pg_controldata.mo
> /usr/share/locale/sl/LC_MESSAGES/pg_resetxlog.mo
> /usr/share/locale/sl/LC_MESSAGES/postgres.mo
> /usr/share/locale/sv/LC_MESSAGES/pg_controldata.mo
> /usr/share/locale/sv/LC_MESSAGES/pg_resetxlog.mo
> /usr/share/locale/sv/LC_MESSAGES/postgres.mo
> /usr/share/locale/ta/LC_MESSAGES/pg_controldata.mo
> /usr/share/locale/ta/LC_MESSAGES/pg_resetxlog.mo
> /usr/share/locale/tr/LC_MESSAGES/pg_controldata.mo
> /usr/share/locale/tr/LC_MESSAGES/pg_resetxlog.mo
> /usr/share/locale/tr/LC_MESSAGES/postgres.mo
> /usr/share/locale/zh_CN/LC_MESSAGES/pg_controldata.mo
> /usr/share/locale/zh_CN/LC_MESSAGES/pg_resetxlog.mo
> /usr/share/locale/zh_CN/LC_MESSAGES/postgres.mo
> /usr/share/locale/zh_TW/LC_MESSAGES/pg_controldata.mo
> /usr/share/locale/zh_TW/LC_MESSAGES/pg_resetxlog.mo
> /usr/share/locale/zh_TW/LC_MESSAGES/postgres.mo
> /usr/share/man/man1/initdb.1.gz
> /usr/share/man/man1/ipcclean.1.gz
> /usr/share/man/man1/pg_controldata.1.gz
> /usr/share/man/man1/pg_ctl.1.gz
> /usr/share/man/man1/pg_resetxlog.1.gz
> /usr/share/man/man1/postgres.1.gz
> /usr/share/man/man1/postmaster.1.gz
> /usr/share/pgsql
> /usr/share/pgsql/conversion_create.sql
> /usr/share/pgsql/information_schema.sql
> /usr/share/pgsql/pg_hba.conf.sample
> /usr/share/pgsql/pg_ident.conf.sample
> /usr/share/pgsql/pg_service.conf.sample
> /usr/share/pgsql/postgres.bki
> /usr/share/pgsql/postgres.description
> /usr/share/pgsql/postgres.shdescription
> /usr/share/pgsql/postgresql.conf.sample
> /usr/share/pgsql/psqlrc.sample
> /usr/share/pgsql/recovery.conf.sample
> /usr/share/pgsql/snowball_create.sql
> /usr/share/pgsql/sql_features.txt
> /usr/share/pgsql/system_views.sql
> /usr/share/pgsql/timezonesets/Africa.txt
> /usr/share/pgsql/timezonesets/America.txt
> /usr/share/pgsql/timezonesets/Antarctica.txt
> /usr/share/pgsql/timezonesets/Asia.txt
> /usr/share/pgsql/timezonesets/Atlantic.txt
> /usr/share/pgsql/timezonesets/Australia
> /usr/share/pgsql/timezonesets/Australia.txt
> /usr/share/pgsql/timezonesets/Default
> /usr/share/pgsql/timezonesets/Etc.txt
> /usr/share/pgsql/timezonesets/Europe.txt
> /usr/share/pgsql/timezonesets/India
> /usr/share/pgsql/timezonesets/Indian.txt
> /usr/share/pgsql/timezonesets/Pacific.txt
> /usr/share/pgsql/tsearch_data/danish.stop
> /usr/share/pgsql/tsearch_data/dutch.stop
> /usr/share/pgsql/tsearch_data/english.stop
> /usr/share/pgsql/tsearch_data/finnish.stop
> /usr/share/pgsql/tsearch_data/french.stop
> /usr/share/pgsql/tsearch_data/german.stop
> /usr/share/pgsql/tsearch_data/hungarian.stop
> /usr/share/pgsql/tsearch_data/hunspell_sample.affix
> /usr/share/pgsql/tsearch_data/ispell_sample.affix
> /usr/share/pgsql/tsearch_data/ispell_sample.dict
> /usr/share/pgsql/tsearch_data/italian.stop
> /usr/share/pgsql/tsearch_data/norwegian.stop
> /usr/share/pgsql/tsearch_data/portuguese.stop
> /usr/share/pgsql/tsearch_data/russian.stop
> /usr/share/pgsql/tsearch_data/spanish.stop
> /usr/share/pgsql/tsearch_data/swedish.stop
> /usr/share/pgsql/tsearch_data/synonym_sample.syn
> /usr/share/pgsql/tsearch_data/thesaurus_sample.ths
> /usr/share/pgsql/tsearch_data/turkish.stop
> /usr/share/pgsql/tsearch_data/xsyn_sample.rules
> /var/lib/pgsql
> /var/lib/pgsql/.bash_profile
> /var/lib/pgsql/backups
> /var/lib/pgsql/data
> [root@vicenza bacula]#
> 
> [root@vicenza bacula]# rpm -ql postgresql-libs
> /usr/lib/libecpg.so.6
> /usr/lib/libecpg.so.6.0
> /usr/lib/libecpg_compat.so.3
> /usr/lib/libecpg_compat.so.3.0
> /usr/lib/libpgtypes.so.3
> /usr/lib/libpgtypes.so.3.0
> /usr/lib/libpq.so.5
> /usr/lib/libpq.so.5.1
> /usr/share/locale/af/LC_MESSAGES/libpq.mo
> /usr/share/locale/cs/LC_MESSAGES/libpq.mo
> /usr/share/locale/de/LC_MESSAGES/libpq.mo
> /usr/share/locale/es/LC_MESSAGES/libpq.mo
> /usr/share/locale/fr/LC_MESSAGES/libpq.mo
> /usr/share/locale/hr/LC_MESSAGES/libpq.mo
> /usr/share/locale/it/LC_MESSAGES/libpq.mo
> /usr/share/locale/ko/LC_MESSAGES/libpq.mo
> /usr/share/locale/nb/LC_MESSAGES/libpq.mo
> /usr/share/locale/pl/LC_MESSAGES/libpq.mo
> /usr/share/locale/pt_BR/LC_MESSAGES/libpq.mo
> /usr/share/locale/ru/LC_MESSAGES/libpq.mo
> /usr/share/locale/sk/LC_MESSAGES/libpq.mo
> /usr/share/locale/sl/LC_MESSAGES/libpq.mo
> /usr/share/locale/sv/LC_MESSAGES/libpq.mo
> /usr/share/locale/ta/LC_MESSAGES/libpq.mo
> /usr/share/locale/tr/LC_MESSAGES/libpq.mo
> /usr/share/locale/zh_CN/LC_MESSAGES/libpq.mo
> /usr/share/locale/zh_TW/LC_MESSAGES/libpq.mo
> [root@vicenza bacula]#
> 
> 
> ####################### Some information on the OS and
> postgresql server
> 
> [root@vicenza bacula]# cat /etc/redhat-release
> Red Hat Enterprise Linux Server release 5.2 (Tikanga)
> 
> 
> [root@vicenza bacula]# rpm -qi postgresql-server
> Name        : postgresql-server            Relocations:
> (not relocatable)
> Version     : 8.3.5                             Vendor:
> (none)
> Release     : 1PGDG.rhel5                   Build Date: Fri
> 31 Oct 2008 
> 02:04:30 PM EDT
> Install Date: Mon 17 Nov 2008 12:01:23 PM EST      Build
> Host: 
> koji-centos5-i386-pg83
> Group       : Applications/Databases        Source RPM: 
> postgresql-8.3.5-1PGDG.rhel5.src.rpm
> Size        : 12406119                         License: BSD
> Signature   : DSA/SHA1, Fri 31 Oct 2008 02:04:48 PM EDT,
> Key ID 
> 1f16d2e1442df0f8
> URL         : http://www.postgresql.org/
> Summary     : The programs needed to create and run a
> PostgreSQL server.
> Description :
> The postgresql-server package includes the programs needed
> to create
> and run a PostgreSQL server, which will in turn allow you
> to create
> and maintain PostgreSQL databases.  PostgreSQL is an
> advanced
> Object-Relational database management system (DBMS) that
> supports
> almost all SQL constructs (including transactions,
> subselects and
> user-defined types and functions). You should install
> postgresql-server if you want to create and maintain your
> own
> PostgreSQL databases and/or your own PostgreSQL server. You
> also need
> to install the postgresql package.
> 
> 
> 
> [root@vicenza bacula]# rpm -qi postgresql-libs
> Name        : postgresql-libs              Relocations:
> (not relocatable)
> Version     : 8.3.5                             Vendor:
> (none)
> Release     : 1PGDG.rhel5                   Build Date: Fri
> 31 Oct 2008 
> 02:04:30 PM EDT
> Install Date: Mon 17 Nov 2008 12:00:07 PM EST      Build
> Host: 
> koji-centos5-i386-pg83
> Group       : Applications/Databases        Source RPM: 
> postgresql-8.3.5-1PGDG.rhel5.src.rpm
> Size        : 556358                           License: BSD
> Signature   : DSA/SHA1, Fri 31 Oct 2008 02:04:36 PM EDT,
> Key ID 
> 1f16d2e1442df0f8
> URL         : http://www.postgresql.org/
> Summary     : The shared libraries required for any
> PostgreSQL clients.
> Description :
> The postgresql-libs package provides the essential shared
> libraries for any
> PostgreSQL client program or interface. You will need to
> install this 
> package
> to use any other PostgreSQL package or any clients that
> need to connect to a
> PostgreSQL server.
> 
> 
> I will keep hunting this down, any comments/tips is really
> appreciated :)
> 
> Pascal S. Clermont
> 
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move
> Developer's challenge
> Build the coolest Linux based applications with Moblin SDK
> & win great prizes
> Grand prize is a trip for two to an Open Source event
> anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Bacula-users mailing list
> Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users


libpq.a is part of postgresql-devel package.


Hemant Shah
E-mail: hjrrs AT yahoo DOT com



      

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users