# Cross Host network communication failure

**URL:** https://forums.suse.com/t/cross-host-network-communication-failure/806
**Category:** Rancher 1.x
**Created:** [October 28, 2015, 5:07pm UTC](https://forums.suse.com/t/cross-host-network-communication-failure/806 "2015-10-28T17:07:41Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![jloisel](https://avatars.discourse-cdn.com/v4/letter/j/74df32/32.png) [@jloisel](https://forums.suse.com/u/jloisel)
#### Post date: [October 28, 2015, 5:07pm UTC](https://forums.suse.com/t/cross-host-network-communication-failure/806/1 "2015-10-28T17:07:42Z")

</div>

Hi,

I opened an issue on GitHub for this but maybe it’s only a configuration issue on my side:

> <https://github.com/rancher/rancher/issues/2466>
>
> Hi,
> I have tried to check all the advice mentioned on http://docs.rancher.com/rancher/faqs/troubleshooting/ page without success. I can't make containers on different hosts...

Basically, i can’t make service links work when services are located on different hosts. For example:

- Zookeeper running on host A,
- Mesos-Master running on host B, with service link to Zookeeper service.

Host A:

- VirtualBox machine create with docker-machine running boot2docker, agent added via custom host registration, 192.168.99.100 as IP being registered.  
Host B:
- VirtualBox machine create with docker-machine running boot2docker, agent added via custom host registration, 192.168.99.101 as IP being registered.

Both VMs are running on my local machine (192.168.0.xx). Rancher server is running on my local machine. It detects both agents and registers them fine. Containers are launched successfully, but somehow they are unable to communicate with each other when running on different hosts.

I tried to go though the documentation and troubleshooting guide with no luck:  
[http://docs.rancher.com/rancher/faqs/troubleshooting/](http://docs.rancher.com/rancher/faqs/troubleshooting/)

Inside the VM, i can ping local containers with 10.42.xx.xx ips, but other VM containers with same network are unreachable.

I checked iptables on both hosts and everything seems fine:  
Chain CATTLE\_PREROUTING (1 references)  
num target prot opt source destination  
1 DNAT udp – 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL udp dpt:4500 to:10.42.191.8:4500  
2 DNAT udp – 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL udp dpt:500 to:10.42.191.8:500

What else can i do to make cross-host networking work? Thanks for your help 🙂

---

<div class="post-metadata">

### Author: ![jloisel](https://avatars.discourse-cdn.com/v4/letter/j/74df32/32.png) [@jloisel](https://forums.suse.com/u/jloisel)
#### Post date: [October 28, 2015, 9:21pm UTC](https://forums.suse.com/t/cross-host-network-communication-failure/806/2 "2015-10-28T21:21:23Z")

</div>

Here are my two VirtualBox hosts: (with bridged network and -e CATTLE\_AGENT\_IP correctly set)

![](https://us1.discourse-cdn.com/flex022/uploads/suse/original/1X/87e9c376fcfdae8d529a485a1f460804ddfaaa30.png)

The Stack is configured as following:

![](https://us1.discourse-cdn.com/flex022/uploads/suse/original/1X/c8c023cf8127946b890161b90c13db71f2ac435e.png)

The rancher server is running on 192.168.0.17. When trying to ping Ubuntu-1 from ubuntu-2:

![](https://us1.discourse-cdn.com/flex022/uploads/suse/original/1X/5ad1012f32ebd1c2cbc099fdcaaaad5b5647d77a.png)

Pinging any 192.168.0.xx host from any VM / local machine works. IPTables seems just fine:

 ![](https://us1.discourse-cdn.com/flex022/uploads/suse/original/1X/a257056509beb4f24fa4267b6fcf9a8a2919bfeb.png)

Tried to disable UFW on local machine (running with Ubuntu 14.04), still no effect.

---

<div class="post-metadata">

### Author: ![jloisel](https://avatars.discourse-cdn.com/v4/letter/j/74df32/32.png) [@jloisel](https://forums.suse.com/u/jloisel)
#### Post date: [October 29, 2015, 6:04pm UTC](https://forums.suse.com/t/cross-host-network-communication-failure/806/3 "2015-10-29T18:04:45Z")

</div>

I could finally manage to get the cross-host network working by trying out with 2 bare-metal machines hosts on the same local area network. It just worked fine.

Setting up hosts on virtualbox is not possible due to NAT network configuration. Even in bridged mode, i couldn’t manage to make them work properly. I managed to run a hybrid configuration with one local host and one AWS EC2 host and it worked fine.

Great job guys, Rancher rocks 😀

---

<div class="post-metadata">

### Author: ![kaos](https://sea2.discourse-cdn.com/flex022/user_avatar/forums.suse.com/kaos/32/38_2.png) [@kaos](https://forums.suse.com/u/kaos)
#### Post date: [October 30, 2015, 6:18am UTC](https://forums.suse.com/t/cross-host-network-communication-failure/806/4 "2015-10-30T06:18:57Z")

</div>

@jloisel, haven’t read your posts, just wanted to say that I’ve used virtualbox without issues, using `docker-machine` to provision the hosts, then standard `docker run`'s to start rancher server and agent. So, it’s most likely something with your setup.

---

<div class="post-metadata">

### Author: ![jloisel](https://avatars.discourse-cdn.com/v4/letter/j/74df32/32.png) [@jloisel](https://forums.suse.com/u/jloisel)
#### Post date: [October 30, 2015, 4:32pm UTC](https://forums.suse.com/t/cross-host-network-communication-failure/806/5 "2015-10-30T16:32:58Z")

</div>

I provisioned two hosts with docker-machine too but i couldn’t manage to get containers communicate between those hosts. By default, it configures both boxes with NAT. I can’t setup NAT for 500/udp and 4500/udp for both machines since it would conflict on my local machine network.

I’ve tried with bridged network and it didn’t work either. I don’t know why, everything seemed fine when inspecting IPTables. Maybe one of my network interfaces on my local machine is messing up the IPSec tunnel.

have you tried making two containers communicate with 2 virtual boxes with NAT networking? I tried to setup NAT port forwarding on both VMs and it didn’t work.

---

<div class="post-metadata">

### Author: ![vkruoso](https://sea2.discourse-cdn.com/flex022/user_avatar/forums.suse.com/vkruoso/32/583_2.png) [@vkruoso](https://forums.suse.com/u/vkruoso)
#### Post date: [November 30, 2015, 11:14pm UTC](https://forums.suse.com/t/cross-host-network-communication-failure/806/6 "2015-11-30T23:14:28Z")

</div>

I`ve the same issues when trying to run more that one VM on the same host. I was trying to get them up using docker-machine and then adding them manually as rancher hosts. At some point linking stops working.

Maybe the Rancher team could clarify the network infrastructure and if that use case is really not supported.

---

<div class="post-metadata">

### Author: ![Julien\_GARCIA\_GONZAL](https://sea2.discourse-cdn.com/flex022/user_avatar/forums.suse.com/julien_garcia_gonzal/32/595_2.png) [@Julien\_GARCIA\_GONZAL](https://forums.suse.com/u/Julien_GARCIA_GONZAL)
#### Post date: [December 1, 2015, 10:32am UTC](https://forums.suse.com/t/cross-host-network-communication-failure/806/7 "2015-12-01T10:32:51Z")

</div>

Hi @kaos,  
I’m trying to make it work with VirtualBox machine provisioned by docker-machine.

The host are visible in Rancher and I can deploy on it, but I cannot link container between those hosts.  
have you tried that?

If it’s working for, i would like to know your setup?  
thanks

---

<div class="post-metadata">

### Author: ![kaos](https://sea2.discourse-cdn.com/flex022/user_avatar/forums.suse.com/kaos/32/38_2.png) [@kaos](https://forums.suse.com/u/kaos)
#### Post date: [December 1, 2015, 12:23pm UTC](https://forums.suse.com/t/cross-host-network-communication-failure/806/8 "2015-12-01T12:23:39Z")

</div>

Ah, right, no I’ve only used two nodes in virtualbox, one for the server and the other a lone agent node. With a dual agent setup, I get the same results as the rest of you.

---

<div class="post-metadata">

### Author: ![jloisel](https://avatars.discourse-cdn.com/v4/letter/j/74df32/32.png) [@jloisel](https://forums.suse.com/u/jloisel)
#### Post date: [December 1, 2015, 8:19pm UTC](https://forums.suse.com/t/cross-host-network-communication-failure/806/9 "2015-12-01T20:19:38Z")

</div>

It’s simply not possible to setup two nodes on virtualbox vms on the same machine unless you configure them with bridged networking. NAT cannot be used since it would require to forward the UDP ports to the local machine. You cannot forward those ports for both the VMs on the same machine.

But, somehow bridged network is not working properly with boot2docker VMs. It freezes my entire Ubuntu setup for an unknown reason.

---

<div class="post-metadata">

### Author: ![Julien\_GARCIA\_GONZAL](https://sea2.discourse-cdn.com/flex022/user_avatar/forums.suse.com/julien_garcia_gonzal/32/595_2.png) [@Julien\_GARCIA\_GONZAL](https://forums.suse.com/u/Julien_GARCIA_GONZAL)
#### Post date: [December 2, 2015, 8:04am UTC](https://forums.suse.com/t/cross-host-network-communication-failure/806/10 "2015-12-02T08:04:48Z")

</div>

On same machine with 2 Virtualbox host, I can managed communication between host, due to same network “[192.168.99.xxx](http://192.168.99.xxx)”.

But cross-multihost not working well.
