Untaint master nodes

[xunholy]

Details

Given some homelabs are using HA master node topology and may not have a large scale cluster the master nodes may require to be untainted. There should be a feature flag to be able to untaint master nodes to allow workloads to be scheduled onto these nodes.

https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#control-plane-node-isolation

[issue-label-bot]

Issue-Label Bot is automatically applying the label feature_request to this issue, with a confidence of 0.92. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[crutonjohn]

Just thinking @xunholy but should this be an on/off switch per cluster or per host?

Shotgun approach would be untainting all masters, making it a cluster-wide function whereas a per-host (perhaps in an inventory-level variable) would allow for a more targeted per-host configuration.

I guess we could also support both somehow but I'm still in the planning phase.

[xunholy]

That is actually a really good question. I think per host is the most configurable and allows people to still keep a single or pair of masters tainted if there is some concern and could be under a more advanced configuration - whilst perhaps having a more generic all or nothing approach is also good to have as a default.

[crutonjohn]

feature compete, to me, would be a per-master untaint

an enhancement would be adding the ability to remove taints from all of them at once

i also believe that i will be adding some sort of warning when deploying with 2 or less worker nodes that notifies the user that if they continue without providing taints that they could run into an issue with running out of resources. additionally, a link to some of our docs that detail the default behavior and why using 1 worker is a bad idea.