Re: [Bacula-users] very long "Dir inserting Attributes"
2014-10-20 11:46:31
job with "Different backup" run on another day. Full jobs not ended at
this time. Early full backup is finish before different is run. Maximum
time for inserting attributes == 2-3 hours. At this time over 10 hours
All full backup is running with hour step for all clients.
20.10.2014 19:18, Bryn Hughes пишет:
> You have a very large number of jobs all trying to do inserts into the
> database at once. What you are probably hitting is the limit of the
> number of transactions/second that your database server can sustain -
> most likely you are I/O bound at the disk level.
>
> All of those 'inserting attribute' jobs are asking for many, many small
> I/O operations (IOPs) - since they are almost certainly not being
> written sequentially this will be causing a lot of seeks on the drive.
> Basically the way you are doing things is probably causing the head on
> the disk to chatter around like crazy.
>
> You will probably find your jobs complete faster if you run fewer at
> once, as right now you are spending most of your time waiting for the
> heads to move on the disks rather than having them actually write anything.
>
> If you run 'iostat -x 2' you will be able to see the %UTL for your
> physical disks (assuming you are on Linux) - you'll probably see your
> disks are in the high 90% if not 100% when all those 'Dir inserting
> Attributes' jobs are running. As I mentioned above, what you are seeing
> is almost certainly the IOP limits of your hardware rather than the
> throughput limits.
>
> Bryn
>
> On 14-10-20 02:14 AM, Anton Gorlov wrote:
>> The problem is observed recently time.
>>
>> After database backup "Dir inserting Attributes" job runs more than 5 hours.
>> Previously, it takes less time. Size of database changed slightly, current
>> size
>> - 85Gb.
>> the loads of io system: iotop read 5 M/s, write several Kb
>> LA 1.4-2.2
>> atop retruns postgres process with bacula INSERT ... argument
>>
>> hdparm -tT /dev/mapper/main-pgsql
>>
>> /dev/mapper/main-pgsql:
>> Timing cached reads: 14626 MB in 2.00 seconds = 7321.24 MB/sec
>> Timing buffered disk reads: 918 MB in 3.03 seconds = 302.72 MB/sec
>>
>>
>>
>> Request list contains:
>> BEGIN; LOCK TABLE Path IN SHARE ROW EXCLUSIVE MODE
>>
>>
>> ===
>> Running Jobs:
>> Console connected at 20-Ок-2014 11:52
>> JobId Level Name Status
>> ======================================================================
>> 8116 Full fluorine.2014-10-19_00.25.00_40 Dir inserting Attributes
>> 8117 Full hydrogen.2014-10-19_00.25.00_41 Dir inserting Attributes
>> 8118 Full neon.2014-10-19_00.32.01_42 Dir inserting Attributes
>> 8122 Full aluminium_DB0.2014-10-19_02.30.00_46 Dir inserting
>> Attributes
>> 8124 Full aluminium_DB5.2014-10-19_02.35.00_48 Dir inserting
>> Attributes
>> 8127 Full aluminium_DB2.2014-10-19_02.50.01_51 Dir inserting
>> Attributes
>> 8129 Differe hydrogen.2014-10-20_00.50.00_57 is waiting on max Client
>> jobs
>> 8130 Differe sulfur_bcp.2014-10-20_01.10.00_01 Dir inserting Attributes
>> 8131 Differe boron_bcp.2014-10-20_02.00.00_03 Dir inserting Attributes
>> 8132 Differe neon.2014-10-20_02.05.00_04 Dir inserting Attributes
>> 8133 Full aluminium_PG5.2014-10-20_02.10.00_05 Dir inserting
>> Attributes
>> 8134 Differe fluorine.2014-10-20_02.20.00_06 Dir inserting Attributes
>> 8135 Full aluminium_DB0.2014-10-20_02.30.00_07 Dir inserting
>> Attributes
>> 8136 Full aluminium_DB4.2014-10-20_02.34.00_08 Dir inserting
>> Attributes
>> 8137 Full aluminium_DB5.2014-10-20_02.35.00_09 Dir inserting
>> Attributes
>> 8138 Full phosphorus_bcp.2014-10-20_02.35.00_10 Dir inserting
>> Attributes
>> 8139 Full aluminium_PG0.2014-10-20_02.35.00_11 Dir inserting
>> Attributes
>> 8140 Full aluminium_PG2.2014-10-20_02.35.00_12 Dir inserting
>> Attributes
>> 8141 Full aluminium_DB2.2014-10-20_02.50.00_13 Dir inserting
>> Attributes
>> 8142 Full aluminium_BILL.2014-10-20_04.35.00_14 Dir inserting
>> Attributes
>> ====
>>
>>
>> postgresql.conf:
>> max_connections = 100
>> shared_buffers = 128MB
>> temp_buffers = 512MB
>> work_mem = 64MB
>> maintenance_work_mem = 32MB
>> fsync = off
>> checkpoint_segments = 90
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Comprehensive Server Monitoring with Site24x7.
>> Monitor 10 servers for $9/Month.
>> Get alerted through email, SMS, voice calls or mobile push notifications.
>> Take corrective actions from your mobile device.
>> http://p.sf.net/sfu/Zoho
>> _______________________________________________
>> Bacula-users mailing list
>> Bacula-users AT lists.sourceforge DOT net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
> ------------------------------------------------------------------------------
> Comprehensive Server Monitoring with Site24x7.
> Monitor 10 servers for $9/Month.
> Get alerted through email, SMS, voice calls or mobile push notifications.
> Take corrective actions from your mobile device.
> http://p.sf.net/sfu/Zoho
> _______________________________________________
> Bacula-users mailing list
> Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://p.sf.net/sfu/Zoho
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
|
|
|