# HA for master nodes
**URL:** https://forums.suse.com/t/ha-for-master-nodes/15442
**Category:** SUSE Rancher Prime
**Created:** [October 4, 2019, 11:14am UTC](https://forums.suse.com/t/ha-for-master-nodes/15442 "2019-10-04T11:14:32Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@kamlesh](https://forums.suse.com/u/kamlesh)
#### Post date: [October 4, 2019, 11:14am UTC](https://forums.suse.com/t/ha-for-master-nodes/15442/1 "2019-10-04T11:14:32Z")
hi, i am very new to rancher and kubernetes. I want to create HA of multiple master nodes using rancher GUI?
please, let me know the procedure.
thanks
---
### Author:  [@superseb](https://forums.suse.com/u/superseb)
#### Post date: [October 4, 2019, 11:40am UTC](https://forums.suse.com/t/ha-for-master-nodes/15442/2 "2019-10-04T11:40:27Z")
HA install for Rancher is documented at [https://rancher.com/docs/rancher/v2.x/en/installation/ha/](https://rancher.com/docs/rancher/v2.x/en/installation/ha/), creating production ready clusters within Rancher is documented at [https://rancher.com/docs/rancher/v2.x/en/cluster-provisioning/production/](https://rancher.com/docs/rancher/v2.x/en/cluster-provisioning/production/)
---
### Author:  [@kamlesh](https://forums.suse.com/u/kamlesh)
#### Post date: [October 7, 2019, 5:04am UTC](https://forums.suse.com/t/ha-for-master-nodes/15442/3 "2019-10-07T05:04:27Z")
just want to confirm that whether Layer 4 load balancer(tcp) works perfectly fine on VM machines? I read somewhere it won’t supports. only layer 7 loadbalancer supports this.
---
### Author:  [@yeti](https://forums.suse.com/u/yeti)
#### Post date: [October 7, 2019, 4:21pm UTC](https://forums.suse.com/t/ha-for-master-nodes/15442/4 "2019-10-07T16:21:53Z")
Create your cluster using RKE, andin your cluster.yaml declare three nodes that are control plane. Its that easy.
Ex:
nodes:
- address: “[yourserver1.dev.yourcompany.com](http://yourserver1.dev.yourcompany.com)”
port: “22”
role: [etcd,controlplane]
user: rancher
- address: “[yourserver2.dev.yourcompany.com](http://yourserver2.dev.yourcompany.com)”
port: “22”
role: [etcd,controlplane]
user: rancher
- address: “[yourserver3.dev.yourcompany.com](http://yourserver3.dev.yourcompany.com)”
port: “22”
role: [etcd,controlplane]
user: rancher
##then declare al your worker nodes
---
### Author:  [@Fraser\_Goffin](https://forums.suse.com/u/Fraser_Goffin)
#### Post date: [October 8, 2019, 7:50am UTC](https://forums.suse.com/t/ha-for-master-nodes/15442/5 "2019-10-08T07:50:28Z")
An L4 load balancer definitely works and is recommended. We deploy to AWS and use an NLB.
---
### Author:  [@kamlesh](https://forums.suse.com/u/kamlesh)
#### Post date: [October 9, 2019, 4:32am UTC](https://forums.suse.com/t/ha-for-master-nodes/15442/6 "2019-10-09T04:32:40Z")
hello yeti,
thanks for immediate reply.
As of now I have created 2 master nodes and 1 worker node.
now i have to test certain cases like :
1. if my first master node gets down, then whether the second master node is able to take entire load? moreover, i need to confirm is it ok to test this with 2 master node or it requires 3 master nodes ?
---
### Author:  [@superseb](https://forums.suse.com/u/superseb)
#### Post date: [October 9, 2019, 9:16am UTC](https://forums.suse.com/t/ha-for-master-nodes/15442/7 "2019-10-09T09:16:15Z")
Please read the documentation linked, in [https://rancher.com/docs/rancher/v2.x/en/cluster-provisioning/production/#count-of-etcd-nodes](https://rancher.com/docs/rancher/v2.x/en/cluster-provisioning/production/#count-of-etcd-nodes) it clearly states 2 etcd nodes does not give you fault tolerance.
---
### Author:  [@yeti](https://forums.suse.com/u/yeti)
#### Post date: [October 9, 2019, 5:24pm UTC](https://forums.suse.com/t/ha-for-master-nodes/15442/8 "2019-10-09T17:24:05Z")
@ [kamlesh] It is generally good practice to always use an **odd** number of masters, as the control-plane nodes perform leader elections.
Leader election is the mechanism that guarantees that only one instance of the kube-scheduler — or one instance of the kube-controller-manager — is actively making decisions, while all the other instances are inactive, but ready to take leadership if something happens to the active one.
---
### Author:  [@JeepGuy](https://forums.suse.com/u/JeepGuy)
#### Post date: [October 11, 2019, 9:58am UTC](https://forums.suse.com/t/ha-for-master-nodes/15442/9 "2019-10-11T09:58:02Z")
> [@yeti](#):
>
> number of masters
THANK YOU !!!
I thought so but all the K8s docs say you only need two Masters… Do you have any reference to validate that the Masters perform leader election?
Jim
---
### Author:  [@javierriera97](https://forums.suse.com/u/javierriera97)
#### Post date: [October 11, 2019, 1:57pm UTC](https://forums.suse.com/t/ha-for-master-nodes/15442/10 "2019-10-11T13:57:28Z")
There you go! [https://rancher.com/docs/rancher/v2.x/en/troubleshooting/kubernetes-resources/#kubernetes-controller-manager-leader](https://rancher.com/docs/rancher/v2.x/en/troubleshooting/kubernetes-resources/#kubernetes-controller-manager-leader)
---
### Author:  [@yeti](https://forums.suse.com/u/yeti)
#### Post date: [October 11, 2019, 2:55pm UTC](https://forums.suse.com/t/ha-for-master-nodes/15442/11 "2019-10-11T14:55:41Z")
[https://medium.com/michaelbi-22303/deep-dive-into-kubernetes-simple-leader-election-3712a8be3a99](https://medium.com/michaelbi-22303/deep-dive-into-kubernetes-simple-leader-election-3712a8be3a99) & others. Just google it
---
### Author:  [@vincent](https://forums.suse.com/u/vincent)
#### Post date: [October 12, 2019, 8:32pm UTC](https://forums.suse.com/t/ha-for-master-nodes/15442/12 "2019-10-12T20:32:13Z")
Words are getting conflated here. There is nothing we call a “master” in Rancher, nodes have the “control plane” or “etcd” role.
etcd has leader election and a "master " inside of itself. You should always have an odd number etcd nodes. There is no reason to ever have an even number except temporarily during a failure or on the way up (or down) to the next odd number; even is strictly worse than odd. And 2 is the absolute worst number to have, because you still have no fault tolerance (if either goes down you have no quorum) but have introduced twice as many hard drives, power supplies, NICs, DIMMs, CPUs etc that could fail.
Control plane nodes talk to etcd, provide the API, and tell worker nodes to do things. More than one provides redundancy in case one fails (and can sometimes horizontally scale load). You do not need an odd number of them. If you have more than one then you need a load balancer or DNS round-robin to distribute requests from users/nodes to the healthy control plane nodes.
---
### Author:  [@kamlesh](https://forums.suse.com/u/kamlesh)
#### Post date: [October 14, 2019, 4:39am UTC](https://forums.suse.com/t/ha-for-master-nodes/15442/13 "2019-10-14T04:39:12Z")
hi,
i am little bit confuse regarding number of control plane and etcd required for HA of master. currently i have updated my cluster with 3 master node (each having 1 etcd role and 1 control plane role) and 1 worker node (which has 1 worker role only).
is it right to move forward ?
or some ground level changes still required before start with installation.
---
### Author:  [@vincent](https://forums.suse.com/u/vincent)
#### Post date: [October 14, 2019, 5:03am UTC](https://forums.suse.com/t/ha-for-master-nodes/15442/14 "2019-10-14T05:03:30Z")
Again, there is nothing called a “master”. To survive the failure of any one node, you want:
- 3 or 5 nodes with the etcd roles
- 2 or more control plane
- 2 or more worker
A single node can have one or more of those roles (i.e. 3 nodes with all 3 roles satisfies the above). Combining etcd and control plane together is common.
---
### Author:  [@kamlesh](https://forums.suse.com/u/kamlesh)
#### Post date: [October 17, 2019, 3:48am UTC](https://forums.suse.com/t/ha-for-master-nodes/15442/15 "2019-10-17T03:48:51Z")
can we put roles (etcd, control plane, and worker ) on the same node? will they work fine ?
---
### Author:  [@Fraser\_Goffin](https://forums.suse.com/u/Fraser_Goffin)
#### Post date: [October 22, 2019, 12:20pm UTC](https://forums.suse.com/t/ha-for-master-nodes/15442/16 "2019-10-22T12:20:44Z")
Yes that will work. You may want to think about the potential consequences though, ie you have less resilience and the possibility that problems with one component will adversely impact the the others. There is also clearly a difference in how you scale this set up if you were to find that any of the components have different resource usage profiles than others (hint, they do).
Anyway, your requirements are your own so that’s what should inform your choices. Technically speaking multi-role nodes are definitely supported.
---
### Author:  [@kamlesh](https://forums.suse.com/u/kamlesh)
#### Post date: [October 29, 2019, 7:47am UTC](https://forums.suse.com/t/ha-for-master-nodes/15442/17 "2019-10-29T07:47:14Z")
hey, i m using this link:
([https://rancher.com/docs/rancher/v2.x/en/installation/ha/](https://rancher.com/docs/rancher/v2.x/en/installation/ha/))
for rancher HA.
here it is mentioned that it is required to install tools namely : RKE, kubectl,helm. As per the doc we are installing kubernetes using RKE. so below are my queries regarding the same :
1. on which nodes (like i have 1 load balancer node, 3 ingress controller nodes, 1 worker node) these tools (RKE, Kubectl, helm) are to be installed?
2. if kubernetes is installed using RKE then is it necessary to install kubectl separately on each node?
---
### Author:  [@Fraser\_Goffin](https://forums.suse.com/u/Fraser_Goffin)
#### Post date: [November 5, 2019, 9:20pm UTC](https://forums.suse.com/t/ha-for-master-nodes/15442/18 "2019-11-05T21:20:30Z")
Those are client side tools so whilst you may choose to install them on your worker of management nodes, more typically you will use whatever your CI/CD platform of choice is to create deployment pipelines that could use helm, vanilla kubectl and rke. Helm 2 is slightly different in the sense that you can install tiller on your nodes. However that’s not a requirement and many people today regard tiller as a security vulnerability (although it is possible to mitigate that in a number of ways). Personally speaking, we have already moved over to Helm 3 which has recently moved to release candidate status.
---
### Author:  [@jpeake](https://forums.suse.com/u/jpeake)
#### Post date: [November 6, 2019, 5:33pm UTC](https://forums.suse.com/t/ha-for-master-nodes/15442/19 "2019-11-06T17:33:11Z")
Also it is important to understand if you are referring to HA for Rancher Server itself (the “local” cluster) or you already have Rnacher running and are creating a workload cluster.
For Rancher HA cluster, you will have all three roles on each node (and should have three nodes, or 5,7,9 if you wanna get crazy). But only Rancher Server runs on this cluster (plus the K8s components)
For a workload cluster managed by Rancher, a common config is 3 nodes with “etc” and “control plane” and then additional nodes with only “worker”.
---
### Author:  [@kamlesh](https://forums.suse.com/u/kamlesh)
#### Post date: [November 14, 2019, 5:21am UTC](https://forums.suse.com/t/ha-for-master-nodes/15442/20 "2019-11-14T05:21:45Z")
hey, thanks guys for your support. 🙂
I have deployed the HA rancher successfully. let me tell you about the cluster that I formed
1. 1 load balancer which is a separated node.
2. 3 ingress nodes (having roles etcd, controlpane) I configured.
3. 1 worker node.
now as per the docs
setup is done successfully, you can view the status of the pods.
[high@loadbalancer creating\_cluster]$ kubectl -n cattle-system get pods
NAME READY STATUS RESTARTS AGE
rancher-85498c4d67-jncjx 1/1 Running 8 7d15h
rancher-85498c4d67-mtvb2 1/1 Running 8 7d15h
rancher-85498c4d67-trmtw 1/1 Running 9 7d15h
\*note i have done below changes
1. disabled and stop firewalld service on all 5 nodes.
2. changed the web port for ngnix from 80 to other random port.
now I need to know how can I open rancher web portal?
i am trying using IP address of one of the ingress controller node.
but getting error : connection refused.
please help.
[Next page](https://forums.suse.com/t/ha-for-master-nodes/15442.md?page=2)