Results 1 to 5 of 5

Thread: Docker Swarm networking

Hybrid View

  1. #1

    Docker Swarm networking

    Hi,

    We are using the Docker distribution installed from the Containers_Module_12_x86_64 package.
    We have containers running locally on the host with mapped ports and everything works fine.

    When we attempt to run the containers as a service under swarm, the service starts up fine but the mapped ports are not exposed on the localhost.
    We have attempted this with very simple services and gotten the same results (including the docker tutorial).
    Example:
    Commands:
    docker swarm init
    docker service create --name my-web --publish published=8080,target=80 --replicas 1 nginx

    Both successful.
    Result from docker service ls
    ID NAME MODE REPLICAS IMAGE PORTS
    ze88zz31le31 my-web replicated 1/1 nginx:latest *:8080->80/tcp

    Result attempting to attach to localhost:8080 where nginx should be accessible
    curl localhost:8080
    curl: (7) Failed to connect to localhost port 8080: Connection refused

    Research on forums seems to indicate that this might be a potential missing dependency in our SLES.
    We are not running SuSEfirewall2 and iptables appears to be populated correctly:
    sudo iptables-save | grep DOCKER
    : DOCKER - [0:0]
    -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
    -A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
    -A DOCKER -i docker0 -j RETURN
    -A DOCKER -i docker_gwbridge -j RETURN
    : DOCKER - [0:0]
    : DOCKER-ISOLATION - [0:0]
    : DOCKER-USER - [0:0]
    -A FORWARD -j DOCKER-USER
    -A FORWARD -j DOCKER-ISOLATION
    -A FORWARD -o docker0 -j DOCKER
    -A FORWARD -o docker_gwbridge -j DOCKER
    -A DOCKER-ISOLATION -i docker_gwbridge -o docker0 -j DROP
    -A DOCKER-ISOLATION -i docker0 -o docker_gwbridge -j DROP
    -A DOCKER-ISOLATION -j RETURN
    -A DOCKER-USER -j RETURN

    I have tried enabling ipv4 forwarding as recommended in a forum, with no seeming effect.
    The docker swarm management ports are available when swarm is running:
    netstat -nlp | grep 7946
    tcp 0 0 :::7946 :::* LISTEN 1828/dockerd
    udp 0 0 :::7946 :::* 1828/dockerd
    netstat -nlp | grep 4789
    udp 0 0 0.0.0.0:4789 0.0.0.0:* -

    I'm at a bit of a dead-end. I read one post that said that LVS / SLES HA Package may be a dependency but I found no corroboration of this.

    Any help would be appreciated.
    Last edited by philu1; 18-Oct-2018 at 23:15. Reason: formatting

  2. Re: Docker Swarm networking

    Hi philu1,
    Quote Originally Posted by philu1 View Post
    Hi,

    We are using the Docker distribution installed from the Containers_Module_12_x86_64 package.
    We have containers running locally on the host with mapped ports and everything works fine.

    When we attempt to run the containers as a service under swarm, the service starts up fine but the mapped ports are not exposed on the localhost.
    [...]
    I'm at a bit of a dead-end. I read one post that said that LVS / SLES HA Package may be a dependency but I found no corroboration of this.

    Any help would be appreciated.
    an online explanation of what you should be seeing network-wise that I've found quite informative can be found at https://neuvector.com/network-securi...er-networking/.

    I'm not sure what tools you might need to add to your current setup, but many cluster and load balancer programs are indeed included in the SLES HAE add-on and your descriptions seem to point to problems creating the ipvs load-balancer setup. But on the other hand, I'd expect to see error messages in the Docker-related logs if such tools were not found while setting up the swarm.

    Regards,
    J
    From the times when today's "old school" was "new school"

    If you find this post helpful and are logged into the web interface, show your appreciation and click on the star below...

  3. #3

    Re: Docker Swarm networking

    Thanks for reference on networking. Here are more details (should have included in edited original post) of docker logs from /var/messages that show a failure in modifying iptables? despite dockerd clearly being able to update iptables.

    Here is a snip of logs that contain an error related to iptables (note that docker appears able to modify iptables per additional info below):
    2018-10-18T19:25:19.287893-04:00 dockerd[1828]: time=“2018-10-18T19:25:19.287650051-04:00” level=info msg=“NetworkDB stats - netID:lxy5sbaswro9qxqa0vt6uq7ob leaving:false netPeers:1 entries:4 Queue qLen:0 netMsg/s:0”
    2018-10-18T19:30:19.487727-04:00 dockerd[1828]: time=“2018-10-18T19:30:19.487514638-04:00” level=info msg=“NetworkDB stats - netID:lxy5sbaswro9qxqa0vt6uq7ob leaving:false netPeers:1 entries:4 Queue qLen:0 netMsg/s:0”
    2018-10-18T19:31:47.425687-04:00 dockerd[1828]: time=“2018-10-18T19:31:47-04:00” level=info msg=“SUSE:secrets :: enabled”
    2018-10-18T19:31:47.540558-04:00 dockerd[1828]: time=“2018-10-18T19:31:47-04:00” level=error msg=“setting up rule failed, [-t mangle -D PREROUTING -p tcp --dport 8080 -j MARK --set-mark 262]: (iptables failed: iptables --wait -t mangle -D PREROUTING -p tcp --dport 8080 -j MARK --set-mark 262: iptables: No chain/target/match by that name.
    2018-10-18T19:31:47.540979-04:00 dockerd[1828]: (exit status 1))”
    2018-10-18T19:31:47.545054-04:00 dockerd[1828]: time=“2018-10-18T19:31:47.542668859-04:00” level=error msg=“Failed to delete firewall mark rule in sbox ingress (ingress): reexec failed: exit status 5”
    2018-10-18T19:31:47.745356-04:00 kernel: [548403.162337] [2768(dockerd)]: gsch_umount_hook_fn(/var/run/docker/netns/cf5394f87151,2) doing
    2
    Last edited by philu1; 19-Oct-2018 at 17:07.

  4. #4

    Re: Docker Swarm networking

    I have now gone through a restart of Docker and Swarm and watched the logs during startup. I found what appear to be the errors at the root of the problem - confirming that it looks like IPVS is required to run Swarm networking:

    2018-10-19T12:23:29.447004-04:00 dockerd[21642]: time="2018-10-19T12:23:29.446754417-04:00" level=warning msg="Running modprobe ip_vs failed with message: `modprobe: WARNING: Module ip_vs not found.`, error: exit status 1"
    2018-10-19T12:23:29.449931-04:00 dockerd[21642]: time="2018-10-19T12:23:29.449047777-04:00" level=error msg="Could not get ipvs family information from the kernel. It is possible that ipvs is not enabled in your kernel. Native loadbalancing will not work until this is fixed."


    The HA package seems quite heavy-weight when ip_vs is the one missing dependency. Is there an alternative to installing the SLES HA package (or is it not as heavy as I perceive)?

    Thanks again.
    Last edited by philu1; 19-Oct-2018 at 17:37.

  5. Re: Docker Swarm networking

    Hi philu1,

    > The HA package seems quite heavy-weight when ip_vs is the one missing dependency. Is there an alternative to installing the SLES HA package (or is it not as heavy as I perceive)?

    Actually, that's more of a licensing question. Because technically, you can limit yourself to only install the according kernel (and its modules) plus whatever else might be needed for ipvs. OTOH, HAE is a non-free SLES add-on.

    Regards,
    J
    From the times when today's "old school" was "new school"

    If you find this post helpful and are logged into the web interface, show your appreciation and click on the star below...

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •