PDA

View Full Version : Network problem? Lots of kernel page allocation failures



tobimat80
24-Dec-2012, 13:36
Hi,

These message appear every second day:


[2012-12-20 07:45:51] The following is only an harmless informational message.
[2012-12-20 07:45:51] Unless you get a _continuous_flood_ of these messages it means
[2012-12-20 07:45:51] everything is working fine. Allocations from irqs cannot be
[2012-12-20 07:45:51] perfectly reliable and the kernel is designed to handle that.
[2012-12-20 07:45:51] postmaster: page allocation failure: order:1, mode:0x20
[2012-12-20 07:45:51] Pid: 5182, comm: postmaster Tainted: G X 3.0.42-0.7-default #1
[2012-12-20 07:45:51] Call Trace:
[2012-12-20 07:45:51] [<ffffffff810048a5>] dump_trace+0x75/0x300
[2012-12-20 07:45:51] [<ffffffff81440073>] dump_stack+0x69/0x6f
[2012-12-20 07:45:51] [<ffffffff810f6e52>] warn_alloc_failed+0x102/0x1a0
[2012-12-20 07:45:51] [<ffffffff810f8909>] __alloc_pages_slowpath+0x549/0x7f0
[2012-12-20 07:45:51] [<ffffffff810f8d99>] __alloc_pages_nodemask+0x1e9/0x200
[2012-12-20 07:45:51] [<ffffffff81139a76>] kmem_getpages+0x56/0x170
[2012-12-20 07:45:51] [<ffffffff8113a8bb>] fallback_alloc+0x19b/0x270
[2012-12-20 07:45:51] [<ffffffff8113b4b7>] kmem_cache_alloc+0x207/0x2b0
[2012-12-20 07:45:51] [<ffffffff81377772>] sk_prot_alloc+0x42/0x190
[2012-12-20 07:45:51] [<ffffffff81378d9a>] sk_clone+0x1a/0x320
[2012-12-20 07:45:51] [<ffffffff813c41a0>] inet_csk_clone+0x10/0xb0
[2012-12-20 07:45:51] [<ffffffff813dd551>] tcp_create_openreq_child+0x21/0x4b0
[2012-12-20 07:45:51] [<ffffffff813dbad7>] tcp_v4_syn_recv_sock+0x47/0x240
[2012-12-20 07:45:51] [<ffffffff813dd3bf>] tcp_check_req+0x2df/0x450
[2012-12-20 07:45:51] [<ffffffff813dc457>] tcp_v4_hnd_req+0x67/0x130
[2012-12-20 07:45:51] [<ffffffff813dc695>] tcp_v4_do_rcv+0x175/0x1e0
[2012-12-20 07:45:51] [<ffffffff8137836c>] release_sock+0x5c/0x120
[2012-12-20 07:45:51] [<ffffffff813c4e1e>] inet_csk_accept+0x8e/0x170
[2012-12-20 07:45:51] [<ffffffff813ebd05>] inet_accept+0x25/0x110
[2012-12-20 07:45:51] [<ffffffff81373087>] sys_accept4+0x177/0x260
[2012-12-20 07:45:51] [<ffffffff8144ac92>] system_call_fastpath+0x16/0x1b
[2012-12-20 07:45:51] [<00007fbbb765b8a0>] 0x7fbbb765b89f

or


[2012-12-21 11:28:29] The following is only an harmless informational message.
[2012-12-21 11:28:29] Unless you get a _continuous_flood_ of these messages it means
[2012-12-21 11:28:29] everything is working fine. Allocations from irqs cannot be
[2012-12-21 11:28:29] perfectly reliable and the kernel is designed to handle that.
[2012-12-21 11:28:29] java: page allocation failure: order:1, mode:0x20
[2012-12-21 11:28:29] Pid: 11018, comm: java Tainted: G X 3.0.42-0.7-default #1
[2012-12-21 11:28:29] Call Trace:
[2012-12-21 11:28:29] [<ffffffff810048a5>] dump_trace+0x75/0x300
[2012-12-21 11:28:29] [<ffffffff81440073>] dump_stack+0x69/0x6f
[2012-12-21 11:28:29] [<ffffffff810f6e52>] warn_alloc_failed+0x102/0x1a0
[2012-12-21 11:28:29] [<ffffffff810f8909>] __alloc_pages_slowpath+0x549/0x7f0
[2012-12-21 11:28:29] [<ffffffff810f8d99>] __alloc_pages_nodemask+0x1e9/0x200
[2012-12-21 11:28:29] [<ffffffff81139a76>] kmem_getpages+0x56/0x170
[2012-12-21 11:28:29] [<ffffffff8113a8bb>] fallback_alloc+0x19b/0x270
[2012-12-21 11:28:29] [<ffffffff8113b4b7>] kmem_cache_alloc+0x207/0x2b0
[2012-12-21 11:28:29] [<ffffffff81377772>] sk_prot_alloc+0x42/0x190
[2012-12-21 11:28:29] [<ffffffff81378d9a>] sk_clone+0x1a/0x320
[2012-12-21 11:28:29] [<ffffffff813c41a0>] inet_csk_clone+0x10/0xb0
[2012-12-21 11:28:29] [<ffffffff813dd551>] tcp_create_openreq_child+0x21/0x4b0
[2012-12-21 11:28:29] [<ffffffff813dbad7>] tcp_v4_syn_recv_sock+0x47/0x240
[2012-12-21 11:28:29] [<ffffffff813dd3bf>] tcp_check_req+0x2df/0x450
[2012-12-21 11:28:29] [<ffffffff813dc457>] tcp_v4_hnd_req+0x67/0x130
[2012-12-21 11:28:29] [<ffffffff813dc695>] tcp_v4_do_rcv+0x175/0x1e0
[2012-12-21 11:28:29] [<ffffffff813dccbb>] tcp_v4_rcv+0x5bb/0x850
[2012-12-21 11:28:29] [<ffffffff813b99c8>] ip_local_deliver_finish+0x128/0x210
[2012-12-21 11:28:29] [<ffffffff81384db2>] __netif_receive_skb+0x622/0x670
[2012-12-21 11:28:29] [<ffffffff81384f03>] process_backlog+0x103/0x200
[2012-12-21 11:28:29] [<ffffffff8138ac1a>] net_rx_action+0x10a/0x2c0
[2012-12-21 11:28:29] [<ffffffff810604bf>] __do_softirq+0xef/0x220
[2012-12-21 11:28:29] [<ffffffff8144bebc>] call_softirq+0x1c/0x30
[2012-12-21 11:28:29] [<ffffffff810043a5>] do_softirq+0x65/0xa0
[2012-12-21 11:28:29] [<ffffffff81060e49>] local_bh_enable_ip+0x99/0xa0
[2012-12-21 11:28:29] [<ffffffff813ebe92>] inet_wait_for_connect+0xa2/0xf0
[2012-12-21 11:28:29] [<ffffffff813ebfe3>] inet_stream_connect+0x103/0x1e0
[2012-12-21 11:28:29] [<ffffffff81372a67>] sys_connect+0xe7/0xf0
[2012-12-21 11:28:29] [<ffffffff8144ac92>] system_call_fastpath+0x16/0x1b
[2012-12-21 11:28:29] [<00007fe7439c7aad>] 0x7fe7439c7aac

I am using kernel 3.0.42-0.7-default #1 SMP Tue Oct 9 11:58:45 UTC 2012 (a8dc443) x86_64 x86_64 x86_64 GNU/Linux

I know it's only a harmless informational message, but it comes up quite often...

Looks like there is something wrong in the TCP stack?

Any ideas?

Thx!

ab
24-Dec-2012, 14:15
Have you tried getting a LAN trace to see, if TCP/IP-related, what may be
causing this?

sudo /usr/sbin/tcpdump -n -s 0 -i any -w /tmp/tcpip.cap

Good luck.

tobimat80
08-Jan-2013, 16:25
Have you tried getting a LAN trace to see, if TCP/IP-related, what may be
causing this?

sudo /usr/sbin/tcpdump -n -s 0 -i any -w /tmp/tcpip.cap

Good luck.

Thanks for the suggestion. But the trace grows very fast - in a few seconds already 2 GB. So this is not feasible as I don't know when this issue will happen again :(

ab
08-Jan-2013, 16:36
If troubleshooting isn't feasible, or at least not worth it, perhaps live
with this "informational message" until something more obvious is wrong.

Good luck.