Page 1 of 2 12 LastLast
Results 1 to 10 of 13

Thread: 10Gb sfp networking troubles HP NC550SFP

Hybrid View

  1. #1

    10Gb sfp networking troubles HP NC550SFP

    So here is the problem, I have recently updated my server hardware to Dell R710s over the holidays. During this time I got a great deal on 3 PCIe HP NC550SFP 10gb network cards. I have purchased the proper HP compatible SFP DAC (direct access copper) cable to connect the 2 systems together.

    After installing SLES 12 and setting it up, I added the 10gb cards in which the OS had no trouble finding. I can configure 1 port on each system with a IP in the 10.10.10.x/24 range.

    p3p1 Link encap:Ethernet HWaddr B4:99:BA:F8B:B8
    UP BROADCAST MULTICAST MTU:1500 Metric:1
    RX packets:0 errors:0 dropped:0 overruns:0 frame:0
    TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:1000
    RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

    p3p2 Link encap:Ethernet HWaddr B4:99:BA:F8B:BC
    UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
    RX packets:0 errors:0 dropped:0 overruns:0 frame:0
    TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:1000
    RX bytes:0 (0.0 b) TX bytes:168 (168.0 b)


    Even though from system "A" I can ping system "A" 10gb network IP, when I try pinging system "B" IP, I get the following....

    p3p1 Link encap:Ethernet HWaddr B4:99:BA:F8:4B8
    inet addr:10.10.10.3 Bcast:10.10.10.255 Mask:255.255.255.0
    UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
    RX packets:0 errors:0 dropped:0 overruns:0 frame:0
    TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:1000
    RX bytes:0 (0.0 b) TX bytes:336 (336.0 b)

    pinky:~ # ping 10.10.10.2
    PING 10.10.10.2 (10.10.10.2) 56(84) bytes of data.
    From 10.10.10.3 icmp_seq=1 Destination Host Unreachable
    From 10.10.10.3 icmp_seq=2 Destination Host Unreachable
    From 10.10.10.3 icmp_seq=3 Destination Host Unreachable


    I could use some assistance in troubleshooting this in hopes of getting 10Gb network between my 2 xen VM servers.

    Thanks,
    John

  2. #2

    Re: 10Gb sfp networking troubles HP NC550SFP

    Johnfm3 wrote:

    > I could use some assistance in troubleshooting this in hopes of
    > getting 10Gb network between my 2 xen VM servers.


    Are both nics connected through a switch?

    Can each of them be pinged from a third device?

    This is purely a guess since I'm unfamiliar your your equipment...

    If they are direct connected, this sounds like what might happneed if
    your devices didn't support Auto-MDIX (automatic medium-dependent
    interface crossover) and you didn't use a crossover cable.

    Or, perhaps your cable is defective...


    --
    Kevin Boyle - Knowledge Partner
    If you find this post helpful and are logged into the web interface,
    please show your appreciation and click on the star below this post.
    Thank you.

  3. #3

    Re: 10Gb sfp networking troubles HP NC550SFP

    Quote Originally Posted by KBOYLE View Post
    Johnfm3 wrote:

    > I could use some assistance in troubleshooting this in hopes of
    > getting 10Gb network between my 2 xen VM servers.


    Are both nics connected through a switch?

    Can each of them be pinged from a third device?

    This is purely a guess since I'm unfamiliar your your equipment...

    If they are direct connected, this sounds like what might happneed if
    your devices didn't support Auto-MDIX (automatic medium-dependent
    interface crossover) and you didn't use a crossover cable.

    Or, perhaps your cable is defective...
    Defective cable is what crossed my mind first

  4. #4

    Re: 10Gb sfp networking troubles HP NC550SFP

    ianrenton wrote:

    > Defective cable is what crossed my mind first


    Are you using a direct connection between your two servers or are you
    using a switch?

    --
    Kevin Boyle - Knowledge Partner
    If you find this post helpful and are logged into the web interface,
    please show your appreciation and click on the star below this post.
    Thank you.

  5. #5

    Re: 10Gb sfp networking troubles HP NC550SFP

    ianrenton wrote:

    > Defective cable is what crossed my mind first


    So you have:
    - one cable?
    - 3 2-port nics?
    - 2 servers?

    If you had second (good) cable you could substitute it to determine if
    the first one is defective.

    Failing that...
    - Are you sure the "active" port on each of the nics are connected?
    - Maybe one of your nics is defective. Have you tried swapping in your
    3rd nic on each server to see if it helps?


    --
    Kevin Boyle - Knowledge Partner
    If you find this post helpful and are logged into the web interface,
    please show your appreciation and click on the star below this post.
    Thank you.

  6. #6

    Re: 10Gb sfp networking troubles HP NC550SFP

    Check the settings and output of ethtools:

    Code:
    # su
    # ethtool eth0
    Settings for eth0:
    	Supported ports: [ TP ]
    	Supported link modes:   10baseT/Half 10baseT/Full 
    	                        100baseT/Half 100baseT/Full 
    	                        1000baseT/Full 
    	Supported pause frame use: No
    	Supports auto-negotiation: Yes
    	Advertised link modes:  10baseT/Half 10baseT/Full 
    	                        100baseT/Half 100baseT/Full 
    	                        1000baseT/Full 
    	Advertised pause frame use: No
    	Advertised auto-negotiation: Yes
    	Speed: 1000Mb/s
    	Duplex: Full
    	Port: Twisted Pair
    	PHYAD: 1
    	Transceiver: internal
    	Auto-negotiation: on
    	MDI-X: on (auto)
    	Supports Wake-on: pumbg
    	Wake-on: g
    	Current message level: 0x00000007 (7)
    			       drv probe link
    	Link detected: yes
    Catch the ping and pong packets with wireshark:

    Code:
    # su
    # dumpcap -i eth0 -B 10 -w /tmp/test.pcapng
    Open /tmp/test.pcapng with wireshark (start wireshark as a normal user, no root rights for security reasons!)

  7. Re: 10Gb sfp networking troubles HP NC550SFP

    Hi John,

    could you please also show the output of "ip addr list" and "ip route list" from both machine A and B, so that we can see any possible effects of your routing setup?

    Regards,
    J
    From the times when today's "old school" was "new school"

    If you find this post helpful and are logged into the web interface, show your appreciation and click on the star below...

  8. #8

    Re: 10Gb sfp networking troubles HP NC550SFP

    I am very sorry to all who have replied. I never received the notification emails and assumed there was no response to my thread. In cleaning up my email junk mail while connected to web mail, I saw a response email.

    Again my deepest apologies.

    As too the problem, I ended up resolving the issue last weekend. The cards firmware were out of date. I updated the BE2NET driver on my systems which caused the system to become unstable but slowed down the boot screen where it shows all the services starting. Slow enough that I saw text showing an error message about the firmware being too old. As such I fought getting the cards updated to the newest firmware, and now I have communication.

    Prior to the firmware, the system would see the cards, and configure the cards. And a system could even ping its own card. But the systems could not ping the other.

    Speeds are currently around the same speeds as my 1G Ethernet with a 20GB file transferring between 2 systems at around 75MB/s and completing in 3min 30sec.

    I have found that the systems look to be running PCIx 2.0 and so there is probably a 5Gb transfer rate capability between cards, as such I will probably never see 10G transfer rate. But for my needs, a 4G to 5G rate would be good enough.


    As too my setup...
    2 Dell Gen 11 R710's (2U)
    each server has a HP NC550SFP dual port network card
    Purchased HP DAC cable for direct connection between the two servers via SFP port.
    Prior to purchasing the hardware, I had confirmed that it was too be Auto-MDIX compliant.

    Since getting this running, I have added a 3rd server to act as a ISCSI storage server with the same card. And the 2 other systems only connect to that machine which has both its ports bridged. This will be my VM environment where the file system holding the VM hd files will be ISCSI.

    Again, thanks again.

    John

  9. #9

    Re: 10Gb sfp networking troubles HP NC550SFP

    Don't forget to tune your system according "System Analysis and Tuning Guide":
    https://www.suse.com/documentation/sles-12/

  10. #10

    Re: 10Gb sfp networking troubles HP NC550SFP

    Does anyone here have access to a good Document on DRBD with LVM for use in XEN vm server configurations? I read thru the NFS server with DRBD and LVM document, but it does not feel like a good fit for my needs.

    The end goal of these 2 servers and the reason for the 10GB network is combining LVM and DRBD where I can pass drive partitions directly to the VMs as harddrives. And have that replicated between the 2 xen servers allowing me to live migrate the VMs between the 2 xen servers.

    In my past setup, the VMs hd's were RAW files, and I was using SLES 11 sp4 and drbd to replicate between 2 disk. But this time, I want to allow xen vms access directly to the hd. I configured a single drive with LVM and gave that to XEN which did a great job of carving partitions out of the overall disk to give to VMs. But now I need that replicated between the servers with DRBD so the vm can live migrate between the 2. Keep in mind, as things change in the VMs environment, it writes those changes to the HD. And those changes have to be replicated between the 2 xen servers in real time. And each xen host will divide up 8 or 10 vms including some plex media vms.

    Meanwhile, my fall back is to run ISCSI which gives me no fail over for my storage and forces my VMs hd's to be RAW file's. This I see is going to cause my performance problems. I am not sure if setting up ISCSI and LVM would be a good setup, nor do able.

    John

Page 1 of 2 12 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •