Good Day!
In our K3S cluster hosten on Hetzner we have a service that produces outbound traffic to reach an external Postgres DB. The database server expects that the IP will stay the same for whitelisting traffic.
To esure that the whitelisting rules should never change, we use Hetzner Flaoting IPs. Effectively attaching a static, reserved IP to any VM.
The network interface eth0 has two public IPs
~ # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host noprefixroute
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
inet <default_vm_IP>/32 brd <default_vm_IP> scope global dynamic eth0
valid_lft 65611sec preferred_lft 65611sec
inet <floating_IP>/32 brd <floating_IP> scope global eth0
For now the service could have such a definition. So every node in the cluster would be whitelisted for the external Postgres DB
apiVersion: v1
kind: Service
metadata:
name: pg-client
namespace: pg-test
spec:
selector:
app: pg-client
type: NodePort
ports:
- name: pg-client
protocol: TCP
port: 5432
targetPort: 5432
nodePort: 32345
The problem is that outgoing traffic always uses default_vm_IP, but it is desired that floating_IP is used.
Reconfiguring K3S has not resulted in the desired effect
curl -sfL https://get.k3s.io | sh -s - agent \
--token=${token} \
--node-ip=$floating_ip \
--server="https://${master_node_ip}:6443" \
--kubelet-arg="cloud-provider=external" \
--node-external-ip="$floating_ip" \
--flannel-iface=$private_network_iface
My next best guess would be to create additional IP tables rules that will force the routing of the pg-client Kubernetes service, to route through that specific floating_ip. But I have no idea how to achieve this.
Any advice on how achieve such a behaviour?
Best Regards