PDA

View Full Version : NFS server error reply Unknown rpc response code with tcpdum



sled1983
02-Oct-2012, 07:52
This is message log error with tcpdump on nfsserver1. (When access shared directory from nfs client)



tcpdump host 192.168.11.6 | grep Unknown


tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
13:26:51.112207 IP nfsserver1.mydomain.nfs > nfsclient1.mydomain.620826288: reply Unknown rpc response code=1406296840 4344
13:26:51.112218 IP nfsserver1.mydomain.nfs > nfsclient1.mydomain.823880773: reply Unknown rpc response code=893448370 4344
13:26:51.112961 IP nfsserver1.mydomain.nfs > nfsclient1.mydomain.620826288: reply Unknown rpc response code=1406296840 4344
13:26:51.112972 IP nfsserver1.mydomain.nfs > nfsclient1.mydomain.823880773: reply Unknown rpc response code=893448370 4344

Some time nfs client is slow. How to monitoring and performance tuning.


nfsserver1
OS: SLES 11 (i586)
Ethernet: 1Gigabit
Networking with Gigabit switch.



26: PCI 200.0: 0200 Ethernet controller
[Created at pci.318]
UDI: /org/freedesktop/Hal/devices/pci_14e4_165a
Unique ID: rBUF.DVt8QtXa57C
Parent ID: gZD2.Ab1ekPN1fM5
SysFS ID: /devices/pci0000:00/0000:00:0b.0/0000:02:00.0
SysFS BusID: 0000:02:00.0
Hardware Class: network
Model: "Broadcom NetXtreme BCM5722 Gigabit Ethernet PCI Express"
Vendor: pci 0x14e4 "Broadcom"
Device: pci 0x165a "NetXtreme BCM5722 Gigabit Ethernet PCI Express"
SubVendor: pci 0x1028 "Dell"
SubDevice: pci 0x0225 "PowerEdge T105 Broadcom NetXtreme 57xx"
Driver: "tg3"
Driver Modules: "tg3"
Device File: eth0
Memory Range: 0xd0200000-0xd020ffff (rw,non-prefetchable)
IRQ: 218 (17931568 events)
HW Address: 00:26:b9:77:7a:c5
Link detected: yes
Module Alias: "pci:v000014E4d0000165Asv00001028sd00000225bc02sc00 i00"
Driver Info #0:
Driver Status: tg3 is active
Driver Activation Cmd: "modprobe tg3"
Config Status: cfg=no, avail=yes, need=no, active=unknown
Attached to: #16 (PCI bridge)


NFS version:
nfs-client-1.1.3-19.2.1
nfs-kernel-server-1.1.3-19.2.1

NFS Status:


nfsstat -s
Server rpc stats:
calls badcalls badauth badclnt xdrcall
1555921 0 0 0 0

Server nfs v3:
null getattr setattr lookup access readlink
11 0% 168367 10% 19709 1% 52648 3% 103970 6% 52 0%
read write create mkdir symlink mknod
599210 38% 414964 26% 12523 0% 1931 0% 83 0% 0 0%
remove rmdir rename link readdir readdirplus
4640 0% 144 0% 5934 0% 497 0% 89 0% 4801 0%
fsstat fsinfo pathconf commit
59892 3% 14 0% 7 0% 102519 6%


Network Status:


netstat -i
Kernel Interface table
Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0 1500 018326204 0 0 018662390 0 0 0 BMRU
lo 16436 0 26829 0 0 0 26829 0 0 0 LRU



nfsclient1 (Linux terminal)
OS: SLED 11 SP1 (x86_64)
Ethernet: 1Gigabit

Users running on this machine around 20 users to used shared data.

NFS version:
nfs-client-1.2.1-2.24.1
nfs-kernel-server-1.2.1-2.24.1

NFS Status:


nfsstat -c
Client rpc stats:
calls retrans authrefrsh
1455064 0 0

Client nfs v3:
null getattr setattr lookup access readlink
0 0% 146686 10% 17582 1% 45519 3% 88724 6% 46 0%
read write create mkdir symlink mknod
580910 39% 388314 26% 11568 0% 1894 0% 83 0% 0 0%
remove rmdir rename link readdir readdirplus
3892 0% 101 0% 5778 0% 410 0% 19 0% 3853 0%
fsstat fsinfo pathconf commit
55604 3% 4 0% 2 0% 104074 7%



Thanks,

sled1983
07-Oct-2012, 12:12
This is message log error with tcpdump on nfsserver1. (When access shared directory from nfs client)



tcpdump host 192.168.11.6 | grep Unknown


tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
13:26:51.112207 IP nfsserver1.mydomain.nfs > nfsclient1.mydomain.620826288: reply Unknown rpc response code=1406296840 4344
13:26:51.112218 IP nfsserver1.mydomain.nfs > nfsclient1.mydomain.823880773: reply Unknown rpc response code=893448370 4344
13:26:51.112961 IP nfsserver1.mydomain.nfs > nfsclient1.mydomain.620826288: reply Unknown rpc response code=1406296840 4344
13:26:51.112972 IP nfsserver1.mydomain.nfs > nfsclient1.mydomain.823880773: reply Unknown rpc response code=893448370 4344

Some time nfs client is slow. How to monitoring and performance tuning.


nfsserver1
OS: SLES 11 (i586)
Ethernet: 1Gigabit
Networking with Gigabit switch.



26: PCI 200.0: 0200 Ethernet controller
[Created at pci.318]
UDI: /org/freedesktop/Hal/devices/pci_14e4_165a
Unique ID: rBUF.DVt8QtXa57C
Parent ID: gZD2.Ab1ekPN1fM5
SysFS ID: /devices/pci0000:00/0000:00:0b.0/0000:02:00.0
SysFS BusID: 0000:02:00.0
Hardware Class: network
Model: "Broadcom NetXtreme BCM5722 Gigabit Ethernet PCI Express"
Vendor: pci 0x14e4 "Broadcom"
Device: pci 0x165a "NetXtreme BCM5722 Gigabit Ethernet PCI Express"
SubVendor: pci 0x1028 "Dell"
SubDevice: pci 0x0225 "PowerEdge T105 Broadcom NetXtreme 57xx"
Driver: "tg3"
Driver Modules: "tg3"
Device File: eth0
Memory Range: 0xd0200000-0xd020ffff (rw,non-prefetchable)
IRQ: 218 (17931568 events)
HW Address: 00:26:b9:77:7a:c5
Link detected: yes
Module Alias: "pci:v000014E4d0000165Asv00001028sd00000225bc02sc00 i00"
Driver Info #0:
Driver Status: tg3 is active
Driver Activation Cmd: "modprobe tg3"
Config Status: cfg=no, avail=yes, need=no, active=unknown
Attached to: #16 (PCI bridge)


NFS version:
nfs-client-1.1.3-19.2.1
nfs-kernel-server-1.1.3-19.2.1

NFS Status:


nfsstat -s
Server rpc stats:
calls badcalls badauth badclnt xdrcall
1555921 0 0 0 0

Server nfs v3:
null getattr setattr lookup access readlink
11 0% 168367 10% 19709 1% 52648 3% 103970 6% 52 0%
read write create mkdir symlink mknod
599210 38% 414964 26% 12523 0% 1931 0% 83 0% 0 0%
remove rmdir rename link readdir readdirplus
4640 0% 144 0% 5934 0% 497 0% 89 0% 4801 0%
fsstat fsinfo pathconf commit
59892 3% 14 0% 7 0% 102519 6%


Network Status:


netstat -i
Kernel Interface table
Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0 1500 018326204 0 0 018662390 0 0 0 BMRU
lo 16436 0 26829 0 0 0 26829 0 0 0 LRU



nfsclient1 (Linux terminal)
OS: SLED 11 SP1 (x86_64)
Ethernet: 1Gigabit

Users running on this machine around 20 users to used shared data.

NFS version:
nfs-client-1.2.1-2.24.1
nfs-kernel-server-1.2.1-2.24.1

NFS Status:


nfsstat -c
Client rpc stats:
calls retrans authrefrsh
1455064 0 0

Client nfs v3:
null getattr setattr lookup access readlink
0 0% 146686 10% 17582 1% 45519 3% 88724 6% 46 0%
read write create mkdir symlink mknod
580910 39% 388314 26% 11568 0% 1894 0% 83 0% 0 0%
remove rmdir rename link readdir readdirplus
3892 0% 101 0% 5778 0% 410 0% 19 0% 3853 0%
fsstat fsinfo pathconf commit
55604 3% 4 0% 2 0% 104074 7%



Thanks,

I'm not sure I understand tcpdump output well. Does this actually mean that
the NFS server is trying to send data to ports beyond 65535? And if yes, do
you have any idea why? Do you think tweaking timeo might help?
http://mail-index.netbsd.org/tech-net/2009/02/15/msg001065.html

performance tuning: I thinking local mounted (/home).