# Overlay network issue

**URL:** <https://forums.suse.com/t/overlay-network-issue/3911>\
**Category:** Rancher 1.x\
**Created:** [September 6, 2016, 10:13pm UTC](https://forums.suse.com/t/overlay-network-issue/3911 "2016-09-06T22:13:52Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![renaudManda](https://sea2.discourse-cdn.com/flex022/user_avatar/forums.suse.com/renaudmanda/32/2444_2.png) [@renaudManda](https://forums.suse.com/u/renaudManda)\
**Post date:** [September 6, 2016, 10:13pm UTC](https://forums.suse.com/t/overlay-network-issue/3911/1 "2016-09-06T22:13:53Z")

</div>

I’m using rancher since several months and I’m very happy with it. Until today I was using rancher to deploy small containers with all services for each slack deployed on a single host.

Now I’d like to connect an LDAP stack to others containers located on different hosts. During my test I discovered that the overlay network and specially the ip addresses are unreachable between hosts !

Here is my context:

- Host (A) with 2 containers 1/ and 2/
- Host (B) with 1 container (LDAP) 3/

1/ 2/ 3/ have an ip address given by rancher overlay network in 10.42.x.x

From 1/ I can ping 2/ (and the reverse work as well) but from 1/ or 2/ I got a timeout when I try to ping 3/

It’s a really simple case, in my production env I’ve 5 actives hosts with 15 containers and I reproduce the trouble on all containers.

How can I troubleshoot a network issue with rancher ?

Thanks for your feedback,  
Renaud

---

<div class="post-metadata">

**Author:** ![vincent](https://sea2.discourse-cdn.com/flex022/user_avatar/forums.suse.com/vincent/32/7156_2.png) [@vincent](https://forums.suse.com/u/vincent)\
**Post date:** [September 7, 2016, 4:29am UTC](https://forums.suse.com/t/overlay-network-issue/3911/2 "2016-09-07T04:29:11Z")

</div>

The most common problems are:

- Each host needs to have a unique IP address (if 2 of them are the same, that’s wrong)
- Each IP needs to be mutually reachable from every other host. The overlay network is direct peer-to-peer communication between the two involved hosts.
- Each host needs 500/udp and 4500/udp open from every other host (as above, those are the ports used for IPSec)

[http://docs.rancher.com/rancher/v1.2/en/faqs/troubleshooting/#cross-host-communication](http://docs.rancher.com/rancher/v1.2/en/faqs/troubleshooting/#cross-host-communication)

---

<div class="post-metadata">

**Author:** ![renaudManda](https://sea2.discourse-cdn.com/flex022/user_avatar/forums.suse.com/renaudmanda/32/2444_2.png) [@renaudManda](https://forums.suse.com/u/renaudManda)\
**Post date:** [September 7, 2016, 8:38am UTC](https://forums.suse.com/t/overlay-network-issue/3911/3 "2016-09-07T08:38:32Z")

</div>

Hello Vincent,

I change my firewall rules to permit these ports, and according to your link it seems that now I’ve another trouble.

Indeed here it’s an extract of my netfilter nat table:  
\> Chain CATTLE\_PREROUTING (1 references)

> num target prot opt source destination  
> 1 MARK tcp – 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL tcp dpt:1639 MARK set 0x668a0  
> 2 DNAT tcp – 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL tcp dpt:1639 to:10.99.139.37:639  
> 3 MARK tcp – 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL tcp dpt:1389 MARK set 0x668a0  
> 4 DNAT tcp – 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL tcp dpt:1389 to:10.99.139.37:389  
> 5 DNAT tcp – 10.99.0.0/16 10.99.0.1 tcp dpt:53 to:169.254.169.250  
> 6 DNAT udp – 10.99.0.0/16 10.99.0.1 udp dpt:53 to:169.254.169.250  
> 7 MARK all – !10.99.0.0/16 169.254.169.250 MAC 02:BD:E1:97:06:AD MARK set 0x272e  
> 8 MARK all – !10.99.0.0/16 169.254.169.250 MAC 02:BD:E1:6E:0D:43 MARK set 0x21f1d  
> 9 MARK all – !10.99.0.0/16 169.254.169.250 MAC 02:BD:E1:E5:E2:68 MARK set 0x1c1f8  
> 10 MARK all – !10.99.0.0/16 169.254.169.250 MAC 02:BD:E1:6A:BA:AB MARK set 0x23e79

So it’s missing:

> 2 DNAT udp – 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL udp dpt:4500 to:10.42.179.222:4500  
> 3 DNAT udp – 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL udp dpt:500 to:10.42.179.222:500

I tried to restart the containers “rancher/agent:v1.0.2” and “rncher/agent-instance:v0.8.3 “/etc/init.d/agent-in” 4 days ago Up 4 minutes 0.0.0.0:500-\>500/udp, 0.0.0.0:4500-\>4500/udp” but the rules still missing…

Sniffing the network and specially the 500 port I can see request form others hosts but without reply coming from my host, which confirm that 500 udp port isn’t forward to the container.

How can I regen the missing rule without doing it by hands ?

Cheers,  
Renaud
