I am starting to see GitOps more and more lately especially now it can be easily installed using an AKS cluster extension. Not sure what a cluster extension is? Check out this blog post. https://pixelrobots.co.uk/2022/02/azure-kubernetes-service-aks-addons-and-extensions-who-is-responsible/

One thing I have noticed is that when people have installed flux via the cluster extension and are using the best practices and tainting your system node pool to only run critical addons the pods are always running on the workload node pools. You may be happy with this, but to me the flux pods are critical to ensure my applications are kept up to date and have no drift.

When looking at the Microsoft docs (https://docs.microsoft.com/en-us/azure/azure-arc/kubernetes/tutorial-use-gitops-flux2) site to see how to install the cluster extension to a node pool with a taint I could not find anything. After having a look at the flux cli install switches (https://fluxcd.io/flux/cmd/flux_install/#examples) I found a switch called --toleration-keys. With this I knew I could pass in this to the cluster extension using the --config option of the az k8s-extension command. Let’s have a look at this in action.

The Command

I won’t go into all the steps to ensure you have the right features and extensions installed for the az cli. You can see all of that in the Microsoft blog post linked above.

The following command will install the flux cluster extension into your cluster and will use the default namespace for flux (flux-system). When you run this command just make sure you change the resource group and cluster name to match yours. In fact, if you have other taints on your node pool you can swap out the "CriticalAddonsOnly=true:NoSchedule" to the one you have.

az k8s-extension create \
	--resource-group rg-aks-pixelland-weu \
	--cluster-name aks-pixelland-weu \
	--name flux \
	--extension-type microsoft.flux \
	--cluster-type managedClusters \
	--auto-upgrade-minor-version true \
	--config toleration-keys="CriticalAddonsOnly=true:NoSchedule"

The beauty of this command is, if you have auto scale enabled on the node pool with the taint it will scale it up to fit the pods on. If you do not have scale up enabled, it will put the pods on another node pool to ensure flux is running.

It’s Microsoft Azure Week!

If we look at one of the deployments YAMLSs under the flux-system namespace we will see the tolerations section is now present.

I hope you found this blog post helpful. If you have any questions or comments, please feel free to reach out.

About the Author

I’m Richard Hooper aka Pixel Robots. I started this blog in 2016 for a couple reasons. The first reason was basically just a place for me to store my step by step guides, troubleshooting guides and just plain ideas about being a sysadmin. The second reason was to share what I have learned and found out with other people like me. Hopefully, you can find something useful on the site.

Reference

Pixel Robots, 2022, Installing Flux (GitOps) AKS cluster extension to a tainted node pool, Pixelrobots.co.uk, Available at: https://pixelrobots.co.uk/2022/09/installing-flux-gitops-aks-cluster-extension-to-a-tainted-node-pool/ [Accessed on 3 July 2023]