kubespray

using kubespray to set up a bare metal k8s cluster

NOTE: Before getting started. SELinux might interfere with Kubernetes if context is not set correctly. It's recommended to disable SELinux for this guide to get Kubernetes installed and working and set SELinux context afterwards if SELinux is needed. Also be aware that certain ports and protocols are required. See documentation at: https://kubernetes.io/docs/reference/networking/ports-and-protocols/. If the cluster is in a private network, it is recommended to disable firewall for the installation and testing.

Run the following commands one by one on a linux terminal. If git or python3 is not installed, do so using system package manager.

git clone https://github.com/kubernetes-sigs/kubespray.git

If ansible is not installed, follow the ansible in a container guide or run the following commands after the git clone command above.

VENVDIR=kubespray-venv
KUBESPRAYDIR=kubespray
python3 -m venv $VENVDIR
source $VENVDIR/bin/activate
cd $KUBESPRAYDIR
pip install -U -r requirements.txt

At this point ansible is ready. Change directory to kubespray if not already there. Copy the sample inventory, I.E. cp -rfp inventory/sample inventory/mycluster. Create a hosts.yaml file in inventory/mycluster with the inventory of the nodes. For this example, there are 3 control plane nodes and 3 worker nodes.

## File hosts.yaml
[all]
cp1 ansible_host=192.168.1.161
cp2 ansible_host=192.168.1.162
cp3 ansible_host=192.168.1.163
w1 ansible_host=192.168.1.164
w2 ansible_host=192.168.1.165
w3 ansible_host=192.168.1.166

[kube_control_plane]
cp1
cp2
cp3

[etcd]
cp1
cp2
cp3

[kube_node]
w1
w2
w3

[k8s_cluster:children]
kube_control_plane
etcd
kube_node

Change the CNI from kubespray's default Calico to Cilium.

• Find kube_network_plugin: key in inventory/mycluster/groups_vars/k8s_cluster/k8s-cluster.yaml and change the value from calico to cilium such that: kube_network_plugin: cilium

• For MetalLB, in the same file, find kube_proxy_strict_arp: false and change it to kube_proxy_strict_arp: true

Depending on how the nodes set up, run: ansible-playbook -i inventory/mycluster/hosts.yaml cluster.yml NOTE: if not familiar with ansible, the easiest would be to copy the root ssh keypair to each of the k8s nodes' root user and run the ansible-playbook command above as root

Once installation is done, run kubectl get nodes and kubectl get all -A to see the nodes and pods in the newly installed kubernetes cluster.

If there are any errors when running the playbook, check that SELinux and firewall are disabled on all the nodes.