Results 1 to 4 of 4

Thread: Strange bonding on SLES11

Hybrid View

  1. #1
    andy s NNTP User

    Strange bonding on SLES11

    I have two servers running SLES11SP1 Xen kernel on S3420GP motherboard
    with 2 integrated Intel LAN interfaces - 82574L and 82578DM configured
    for LACP 802.3ad bonding. The switch ports are also configured for LACP,
    and port statistics show that one port is mostly sending, another
    mostly receiving packets on both severs. So I feel it as a kind of load
    The third SLES11SP1 server is running normal kernel and is configured
    as iSCSI target for both Xen servers. It has SuperMicro X7SPA-H-O
    motherboard with 2x Intel 82574L Gigabit LAN interfaces also configured
    for LACP 802.3ad bonding. But switch port statistics show that one port
    is nearly idle the whole time while another is loaded with both RX and
    TX packets. No load balancing of any kind because only one port is
    If I disconnect the busy port, load goes to the idle one, so connection
    is fault tolerant.

    Is it possible to push anyhow the load balancing on this bond?

    andy_s's Profile:
    View this thread:

  2. #2
    Bob-O-Rama NNTP User

    Re: Strange bonding on SLES11

    It may be that 802.3ad "Working As Designed, Design Is Bad" in that with
    802.3ad, it ensure that > "All packets associated with a given 'conversation' are
    > transmitted on the same link to prevent mis-ordering." So could it be there the traffic is just divided unequally? In

    other words with a small number of flows the division of labor could be
    very unequal. Note also that this does not provide a solution for a
    single flow needing both links worth of bandwidth.

    -- Bob

    Bob Mahar -- Novell Knowledge Partner
    Do you do what you do at a .EDU?
    "Programming is like teaching a jellyfish to build a house."
    More Bob: 'Twitter' ( 'Blog'
    ( 'Vimeo' ( <--
    Click And Be Amazed!
    Bob-O-Rama's Profile:
    View this thread:

  3. #3
    andy s NNTP User

    Re: Strange bonding on SLES11

    This idea is clear.

    But I went forward with experimenting. I made NFS shares on servers
    with both types of motherboard (Intel and SuperMicro). Then I started
    file copying (two files ~4GB each) on those shares from two Linux boxes
    simultaneously in 3 modes:
    1. Both reading
    2. Both writing
    3. One - reading, another - writing.

    The results were strange again:
    1. On both motherboards two simultaneous reading or writing operations
    were served from the same port (no balancing between two different
    connections to Linux boxes).
    2. On Intel motherboard (2 different LAN chipsets) simultaneous reading
    was through one LAN port, but simultaneous writing was through another.
    3. On SuperMicro motherboard (2 same LAN chipsets) both simultaneous
    reading and simultaneous writing were through one LAN port, another was
    nearly idle (~300 B/s).
    4. In one box reading, another - writing mode, Intel motherboard used 2
    ports, but SuperMicro - again only one for both boxes.

    Any ideas?

    andy_s's Profile:
    View this thread:

  4. #4
    andy s NNTP User

    Re: Strange bonding on SLES11

    I found good advice while googling the item:
    ad_select=bandwidth xmit_hash_policy=layer2+3

    This killed any difference in motherboard behavior - both Intel and
    SuperMicro now look the same.
    Nice balancing for parallel reading or writing - both interfaces are
    But when one client is reading and another - writing, the traffic from
    both of them goes through one LAN interface.

    andy_s's Profile:
    View this thread:


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts