>>>>> On Tue, 29 May 2012 16:20:18 +0200, Uwe Schuerkamp said:
>
> Hi folks,
>
> recently we've been seeing more and more problems with bacula-fd
> messages in dmesg about a page allocation failure.
>
> Platform is centos 6.2 64 bit, Version 5.2.6 compiled from Source
> using the stock distro gcc.
>
> We're using MariaDB 5.x as the db backend, here are some stats about
> the bacula installation itself:
>
> Total clients: 128 Total bytes stored: 77.33 TB
> Total files: 76905116 Database size: 111.72 GB
>
> The server has 18G RAM, backup performance is generally rather good.
>
> Online backups are going to disk (33TB full of 39TB, xfs based FS).
>
> Here's the message from dmesg:
>
> May 28 12:24:16 bacula-server kernel: bacula-sd: page allocation failure.
> order:1, mode:0x20
> May 28 12:24:16 bacula-server kernel: Pid: 21923, comm: bacula-sd Not tainted
> 2.6.32-71.29.1.el6.x86_64 #1
> May 28 12:24:16 bacula-server kernel: Call Trace:
> May 28 12:24:16 bacula-server kernel: <IRQ>
> [<ffffffff8111eab6>]__alloc_pages_nodemask+0x706/0x850
> May 28 12:24:16 bacula-server kernel: [<ffffffff81156212>]
> kmem_getpages+0x62/0x170
> May 28 12:24:16 bacula-server kernel: [<ffffffff81156e2a>]
> fallback_alloc+0x1ba/0x270
> May 28 12:24:16 bacula-server kernel: [<ffffffff8115687f>]
> ?cache_grow+0x2cf/0x320
> May 28 12:24:16 bacula-server kernel: [<ffffffff81156ba9>]
> ____cache_alloc_node+0x99/0x160
> May 28 12:24:16 bacula-server kernel: [<ffffffff8115750b>]
> kmem_cache_alloc+0x11b/0x190
> May 28 12:24:16 bacula-server kernel: [<ffffffff81404448>]
> sk_prot_alloc+0x48/0x180
> May 28 12:24:16 bacula-server kernel: [<ffffffff81404692>] sk_clone+0x22/0x2a0
> May 28 12:24:16 bacula-server kernel: [<ffffffff8144c276>]
> inet_csk_clone+0x16/0xd0
> May 28 12:24:16 bacula-server kernel: [<ffffffff814651c3>]
> tcp_create_openreq_child+0x23/0x450
> May 28 12:24:16 bacula-server kernel: [<ffffffff81462c0d>]
> tcp_v4_syn_recv_sock+0x4d/0x280
> May 28 12:24:16 bacula-server kernel: [<ffffffff81464f81>]
> tcp_check_req+0x201/0x420
> May 28 12:24:16 bacula-server kernel: [<ffffffff8146262b>]
> tcp_v4_do_rcv+0x35b/0x430
> May 28 12:24:16 bacula-server kernel: [<ffffffff8105c484>]
> ?try_to_wake_up+0x284/0x380
> May 28 12:24:16 bacula-server kernel: [<ffffffff81463e40>]
> tcp_v4_rcv+0x5b0/0x7e0
> May 28 12:24:16 bacula-server kernel: [<ffffffff8105c592>]
> ?default_wake_function+0x12/0x20
> May 28 12:24:16 bacula-server kernel: [<ffffffff81441e7d>]
> ip_local_deliver_finish+0xdd/0x2d0
> May 28 12:24:16 bacula-server kernel: [<ffffffff81442108>]
> ip_local_deliver+0x98/0xa0
> May 28 12:24:16 bacula-server kernel: [<ffffffff814415cd>]
> ip_rcv_finish+0x12d/0x440
> May 28 12:24:16 bacula-server kernel: [<ffffffff81441b55>] ip_rcv+0x275/0x350
> May 28 12:24:16 bacula-server kernel: [<ffffffff8140ffeb>]
> netif_receive_skb+0x38b/0x670
> May 28 12:24:16 bacula-server kernel: [<ffffffff8126ce48>]
> ?is_swiotlb_buffer+0x18/0x50
> May 28 12:24:16 bacula-server kernel: [<ffffffffa0269238>]
> bnx2_poll_work+0xd18/0x1240 [bnx2]
> May 28 12:24:16 bacula-server kernel: [<ffffffff8134a57a>]
> ?scsi_next_command+0x4a/0x60
> May 28 12:24:16 bacula-server kernel: [<ffffffff8134b36e>]
> ?scsi_io_completion+0x35e/0x550
> May 28 12:24:16 bacula-server kernel: [<ffffffff8105c846>]
> ?update_curr+0xe6/0x1e0
> May 28 12:24:16 bacula-server kernel: [<ffffffffa026979d>]
> bnx2_poll_msix+0x3d/0xc0 [bnx2]
> May 28 12:24:16 bacula-server kernel: [<ffffffff81410b73>]
> net_rx_action+0x103/0x210
> May 28 12:24:16 bacula-server kernel: [<ffffffff81073d67>]
> __do_softirq+0xb7/0x1e0
> May 28 12:24:16 bacula-server kernel: [<ffffffff810d8a10>]
> ?handle_IRQ_event+0x60/0x170
> May 28 12:24:16 bacula-server kernel: [<ffffffff81073dc4>]
> ?__do_softirq+0x114/0x1e0
> May 28 12:24:16 bacula-server kernel: [<ffffffff810142cc>]
> call_softirq+0x1c/0x30
> May 28 12:24:16 bacula-server kernel: [<ffffffff81015f35>]
> do_softirq+0x65/0xa0
> May 28 12:24:16 bacula-server kernel: [<ffffffff81073b65>] irq_exit+0x85/0x90
> May 28 12:24:16 bacula-server kernel: [<ffffffff814d0945>] do_IRQ+0x75/0xf0
> May 28 12:24:16 bacula-server kernel: [<ffffffff81013ad3>]
> ret_from_intr+0x0/0x11
>
> Any idea what's going wrong here? I don't see any significant swapping
> or memory usage when this happens. I can provide a full dmesg dump if
> that's helpful, I just didn't want to spam the list beyond measure
> this time.
Looks like a kernel or driver issue to me, not a problem with bacula-sd. You
could ask on the mailing list.
__Martin
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
|
