# HA with scheduling

**URL:** <https://forums.suse.com/t/ha-with-scheduling/1252>\
**Category:** Rancher 1.x\
**Created:** [December 15, 2015, 10:57am UTC](https://forums.suse.com/t/ha-with-scheduling/1252 "2015-12-15T10:57:08Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Swaroop\_Kundeti](https://avatars.discourse-cdn.com/v4/letter/s/dfb087/32.png) [@Swaroop\_Kundeti](https://forums.suse.com/u/Swaroop_Kundeti)\
**Post date:** [December 15, 2015, 10:57am UTC](https://forums.suse.com/t/ha-with-scheduling/1252/1 "2015-12-15T10:57:08Z")

</div>

Hello There,

Though there is no option for hot/live migrations for docker. I would like to use scheduling as an option for migrating containers.

Example:

I label all my hosts with “status=healthy” and spin up container on a condition with must option setting to scheduling label “status=healthy”. Do containers migrate to healthy instance? If the label of host is changed to “status=unhealthy”

Also I’m using flocker for managing volume so there is no worries about volumes.

Swaroop Kundeti  
Site Reliability Engineer • Making Happiness Possible  
Voice: +91-9866221131, Skype : swaroop.kundeti  
Follow Axelerant on F T L G G+

---

<div class="post-metadata">

**Author:** ![denise](https://avatars.discourse-cdn.com/v4/letter/d/82dd89/32.png) [@denise](https://forums.suse.com/u/denise)\
**Post date:** [December 18, 2015, 2:08am UTC](https://forums.suse.com/t/ha-with-scheduling/1252/2 "2015-12-18T02:08:55Z")

</div>

If the host label is removed, the containers will not get re-scheduled. There is a Github issue that would allow the removal of containers based on label scheduling.

[https://github.com/rancher/rancher/issues/2678](https://github.com/rancher/rancher/issues/2678)

---

<div class="post-metadata">

**Author:** ![RVN\_BR](https://sea2.discourse-cdn.com/flex022/user_avatar/forums.suse.com/rvn_br/32/2842_2.png) [@RVN\_BR](https://forums.suse.com/u/RVN_BR)\
**Post date:** [December 31, 2015, 7:52am UTC](https://forums.suse.com/t/ha-with-scheduling/1252/3 "2015-12-31T07:52:02Z")

</div>

Will there be any scheduling based on system resources/best host for the job?

What about for geolocation? (labels could be one thing but it wouldnt provide for fallback)…

thanks

---

<div class="post-metadata">

**Author:** ![denise](https://avatars.discourse-cdn.com/v4/letter/d/82dd89/32.png) [@denise](https://forums.suse.com/u/denise)\
**Post date:** [December 31, 2015, 5:36pm UTC](https://forums.suse.com/t/ha-with-scheduling/1252/4 "2015-12-31T17:36:55Z")

</div>

There is a Github request to schedule based on resources.

> <https://github.com/rancher/rancher/issues/2228>
>
> I know the Rancher documentation clearly states that scheduling based on host resources is not supported.
> Here's the feature as described in...

As for fallback, I think you this request might help.

> <https://github.com/rancher/rancher/issues/2247>
>
> At present you have the options to schedule where a services containers run based upon either host labels or container labels...

---

<div class="post-metadata">

**Author:** ![RVN\_BR](https://sea2.discourse-cdn.com/flex022/user_avatar/forums.suse.com/rvn_br/32/2842_2.png) [@RVN\_BR](https://forums.suse.com/u/RVN_BR)\
**Post date:** [January 26, 2016, 8:08pm UTC](https://forums.suse.com/t/ha-with-scheduling/1252/5 "2016-01-26T20:08:04Z")

</div>

Further on this topic, is there any plans for container “re-scheduling”? i.e. if I have 1 host and start services which have soft anti-affinity rules with themselves at a scale =2, Host1 gets 2 copies of the service… This is correct as the anti-affinity rules are soft…

However, if I add a host, the services dont ever change to the other host… 2 things come to mind, simple “start then stop” from one host to another, all the way to “live migration” of continaers, which I understand is already a feature of docker but still hasnt seen its way into rancher…

Any idea on these two fronts? I guess some containers would be good to have “start-then-stop” type mechanisms, while others may be crucial to maintain the container running… This could maybe be defined somewhere… anyways, I’ve noticed that if I restart the container it wont move it, so I gather its just not taken into considueration at all currently…?

---

<div class="post-metadata">

**Author:** ![demarant](https://sea2.discourse-cdn.com/flex022/user_avatar/forums.suse.com/demarant/32/497_2.png) [@demarant](https://forums.suse.com/u/demarant)\
**Post date:** [January 26, 2016, 9:03pm UTC](https://forums.suse.com/t/ha-with-scheduling/1252/6 "2016-01-26T21:03:09Z")

</div>

@RVN_BR what you are referring to is exactly what I have also mentioned in the issue [on github about rebalancing](https://github.com/rancher/rancher/issues/2558#issuecomment-170033560)

so you could +1 that one 🙂

There is currently no way with rancher scheduling to guarantee that a Stack is on at least two hosts. So that one can easily do docker engine upgrades/hosts replacements. Rancher will just reschedule containers if they are found down/unhealthy but no guarantee they will end up spread evenly with soft anti-affinity (and hard-affinity will only put one on each host). I hope Rancher will address this in the future, because right now we have to do silly tricks like “splitting services” or creating two identical Stacks and make sure one Stack is not deployed on the same hosts as the other Stack.

[this issue was also very much related](https://github.com/rancher/rancher/issues/3379) but got closed in favor of another more complex one.

---

<div class="post-metadata">

**Author:** ![RVN\_BR](https://sea2.discourse-cdn.com/flex022/user_avatar/forums.suse.com/rvn_br/32/2842_2.png) [@RVN\_BR](https://forums.suse.com/u/RVN_BR)\
**Post date:** [January 26, 2016, 9:05pm UTC](https://forums.suse.com/t/ha-with-scheduling/1252/7 "2016-01-26T21:05:40Z")

</div>

I’m going to check and +1 your github issue, but if a soft anti-affinity rule were respected, it would save you from having to do other kinds of “workarounds” such as splitting services, etc, no?

---

<div class="post-metadata">

**Author:** ![demarant](https://sea2.discourse-cdn.com/flex022/user_avatar/forums.suse.com/demarant/32/497_2.png) [@demarant](https://forums.suse.com/u/demarant)\
**Post date:** [January 26, 2016, 9:10pm UTC](https://forums.suse.com/t/ha-with-scheduling/1252/8 "2016-01-26T21:10:39Z")

</div>

unfortunately not, in my tests soft-anti affinity only works when you just start with fresh hosts (empty hosts) or hosts with equal number of containers on them. As soon as you have hosts with a different numbers of containers, you can end up a stack in a single host. I will try to post an exact compose file to show how to reproduce it.  
you can read the entire post, i was quite detailed in there

---

<div class="post-metadata">

**Author:** ![RVN\_BR](https://sea2.discourse-cdn.com/flex022/user_avatar/forums.suse.com/rvn_br/32/2842_2.png) [@RVN\_BR](https://forums.suse.com/u/RVN_BR)\
**Post date:** [January 27, 2016, 12:29pm UTC](https://forums.suse.com/t/ha-with-scheduling/1252/9 "2016-01-27T12:29:02Z")

</div>

Hi @demarant I agree with you, and I saw the same effects where for some reason it starts scheduling containers on a single host independent of affinity rules… I posted here somewhere… I will try to find the link soon as I’m mobile right now…

Eitherway what you mention is correct, and my comment was basically that the anti-affinity and other scheduling rules should be reviewed, not that they are working 😉 Re-scheduling should also be part of that… But I’m agreeing with you 🙂

---

<div class="post-metadata">

**Author:** ![denise](https://avatars.discourse-cdn.com/v4/letter/d/82dd89/32.png) [@denise](https://forums.suse.com/u/denise)\
**Post date:** [February 1, 2016, 11:16pm UTC](https://forums.suse.com/t/ha-with-scheduling/1252/10 "2016-02-01T23:16:13Z")

</div>

Technically, soft affinity rules are working as at the time that you are scheduling your service, it’s honoring your soft affinity rules. Unless you have an exact example that shows me what about soft affinity rules are broken. Also, please remember that it’s soft rules, so technically anything soft could be ignored.

@RVN_BR I can’t remember if your issue is the one that I’m looking at that has soft affinity rules and sidekicks involved. But if so, that’s a more complicated case as sidekicks and links have dependencies on each other of who starts first. I still believe that soft affinity rules are working.

---

<div class="post-metadata">

**Author:** ![RVN\_BR](https://sea2.discourse-cdn.com/flex022/user_avatar/forums.suse.com/rvn_br/32/2842_2.png) [@RVN\_BR](https://forums.suse.com/u/RVN_BR)\
**Post date:** [February 2, 2016, 4:00pm UTC](https://forums.suse.com/t/ha-with-scheduling/1252/11 "2016-02-02T16:00:30Z")

</div>

Hi @denise, I stopped using sidekicks as I was having more issues and ended up just sticking everything in the containers manually ☹ Ideally I’d like to go back to that…

I reported somewhere here about an issue with using variables for stack/service in the affinity rules… I’ll look for that post… (EDIT: here it is [What is the service hash / is it important & some other rancher-compose questions?](http://forums.suse.com/t/what-is-the-service-hash-is-it-important-some-other-rancher-compose-questions/1477/8) )
