# Unknown failure in multihost loadtest **URL:** \ **Category:** Rancher 1.x\ **Created:** [October 14, 2015, 9:24am UTC](https://forums.suse.com/t/unknown-failure-in-multihost-loadtest/699 "2015-10-14T09:24:35Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Flo\_B](https://avatars.discourse-cdn.com/v4/letter/f/f1d935/32.png) [@Flo\_B](https://forums.suse.com/u/Flo_B)\ **Post date:** [October 14, 2015, 9:24am UTC](https://forums.suse.com/t/unknown-failure-in-multihost-loadtest/699/1 "2015-10-14T09:24:35Z")
Hi Rancher Labs, i made a few loadtest with this script and saw some behaviour in the infrastructure i cannot explain. ![](https://us1.discourse-cdn.com/flex022/uploads/suse/original/1X/3a13ac26e5a4ef4dbc96af1bec45a1be52748dc6.PNG) . . . . ![](https://us1.discourse-cdn.com/flex022/uploads/suse/original/1X/bd9f24348839fd9f2b1e66cd5a4ce0c4f842e718.PNG) . . . ![](https://us1.discourse-cdn.com/flex022/uploads/suse/original/1X/9e6e60924488d3a6f39ec34cc6556c638998be7c.PNG) . . . now i run this “loadtest script” on a 3rd VM. i=“0” while [$i -lt 100000] do echo “versuch $1” wget [http://10.8.31.150/4K.txt](http://10.8.31.150/4K.txt) rm -f 4K.txt sleep 0.01 i=$[$i+1] done After a couple of seconds (10-30) i just reveice a 503 with “no servers available to handle the request”. You can also run this one: ab -n 100000 -c 32 [http://10.8.31.150:80/4K.txt](http://10.8.31.150:80/4K.txt) Then the 503 comes after 3-5 seconds… At that point my lb is not able to ping the 10.42.244.70 (nginx) anymore. And now i’m going to wait some minutes (5-10min) and my lb is able to ping 10.42.244.70 again… If i run the test again, it is failing after some seconds again… Could you please repeat that test in your Infrastructure? Maybe i got a bug in my infrastructure… Or it’s in the multihost-networking implementation… i hope it’s the first one 😉 Regards, Florian ---
**Author:** ![Flo\_B](https://avatars.discourse-cdn.com/v4/letter/f/f1d935/32.png) [@Flo\_B](https://forums.suse.com/u/Flo_B)\ **Post date:** [October 14, 2015, 11:35am UTC](https://forums.suse.com/t/unknown-failure-in-multihost-loadtest/699/2 "2015-10-14T11:35:59Z")
Hey, i just set up a 2nd environment on a completly different Infrastructure (two laptops). the result ist the same. After a couple of seconds the LB is repsonding a 503 (no servers available) and the lb isn’t able to ping the nginx container on the other laptop. 5-10minutes of waiting the ping is working again… If i just test on 1 host it’s working fine… but of course i want to use rancher in a multi host szenario… I’m using CentOs 7 as base-image for the hosts. Maybe it’s an issue about that… im going to test it again under ubuntu 14.04 server. ---
**Author:** ![vincent](https://sea2.discourse-cdn.com/flex022/user_avatar/forums.suse.com/vincent/32/7156_2.png) [@vincent](https://forums.suse.com/u/vincent)\ **Post date:** [October 14, 2015, 4:51pm UTC](https://forums.suse.com/t/unknown-failure-in-multihost-loadtest/699/3 "2015-10-14T16:51:06Z")
This sounds like you’re probably running out of ephemeral ports in the balancer container because it’s a large number of short requests coming from a single source… Here’s a decent explanation: [https://www.box.com/blog/ephemeral-port-exhaustion-and-web-services-at-scale/](https://www.box.com/blog/ephemeral-port-exhaustion-and-web-services-at-scale/) I do not think the balancer container currently has any tuning for this, but it should… ---
**Author:** ![vincent](https://sea2.discourse-cdn.com/flex022/user_avatar/forums.suse.com/vincent/32/7156_2.png) [@vincent](https://forums.suse.com/u/vincent)\ **Post date:** [October 14, 2015, 4:54pm UTC](https://forums.suse.com/t/unknown-failure-in-multihost-loadtest/699/4 "2015-10-14T16:54:49Z")
Or apparently there’s a similar sort of issue with the IPSec routing table for the overlay network, which is probably more likely since you said a single host works fine. There’s a fix for that in the next release. ---
**Author:** ![Flo\_B](https://avatars.discourse-cdn.com/v4/letter/f/f1d935/32.png) [@Flo\_B](https://forums.suse.com/u/Flo_B)\ **Post date:** [October 15, 2015, 7:08am UTC](https://forums.suse.com/t/unknown-failure-in-multihost-loadtest/699/5 "2015-10-15T07:08:13Z")
Hey vincent, thank you for your fast response. When is the release date? Could you reconstruct my problem? Would be really nice, if that would be fixed. If you want to go to production in high-load applications it’s a must i guess… Regards, Florian ---
**Author:** ![denise](https://avatars.discourse-cdn.com/v4/letter/d/82dd89/32.png) [@denise](https://forums.suse.com/u/denise)\ **Post date:** [October 22, 2015, 8:12pm UTC](https://forums.suse.com/t/unknown-failure-in-multihost-loadtest/699/6 "2015-10-22T20:12:39Z")
There is a fix with the ipsec in v0.42.0. Could you test with that version to see if it fixes your issue. It fixed another person’s issues that were similar. ---
**Author:** ![Flo\_B](https://avatars.discourse-cdn.com/v4/letter/f/f1d935/32.png) [@Flo\_B](https://forums.suse.com/u/Flo_B)\ **Post date:** [November 2, 2015, 10:13am UTC](https://forums.suse.com/t/unknown-failure-in-multihost-loadtest/699/7 "2015-11-02T10:13:29Z")
Hi denise, thank you for the update. It’s working now. Thank you for the fix! Regards, Florian ---
**Author:** ![denise](https://avatars.discourse-cdn.com/v4/letter/d/82dd89/32.png) [@denise](https://forums.suse.com/u/denise)\ **Post date:** [November 2, 2015, 6:15pm UTC](https://forums.suse.com/t/unknown-failure-in-multihost-loadtest/699/8 "2015-11-02T18:15:15Z")