K8s Installation Guide

This page will walk you through the installation process of the Kubernetes cluster with 1 Master Node and 3 Worker Nodes on RHEL/Rocky/Centos Linux Distro.

Installation Steps for Master Nodes

Step 1: Create 4px user

  • Create 4px user

sudo useradd 4px
sudo passwd 4px

  • Add 4px user to the sudoers file

sudo visudo
4px ALL=(ALL) NOPASSWD: ALL #Add this line to sudoers file

  • Switch to 4px user

sudo su - 4px

Step 2 Install Docker:

  • Configure the yum repo to install the docker
    sudo yum install -y yum-utils
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
  • Install and configure the docker
    sudo yum install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo usermod -aG docker $USER
newgrp docker
  • Start and enable the docker service
    sudo systemctl start docker
sudo systemctl enable docker.service
sudo systemctl enable containerd.service

Step 3: Install Kubernetes

  • Configure the yum repo to install KubernetesNote: This overwrites any existing configuration in /etc/yum.repos.d/kubernetes.repo
    cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/v1.29/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/v1.29/rpm/repodata/repomd.xml.key
exclude=kubelet kubeadm kubectl cri-tools kubernetes-cni
EOF
  • Disable SELinux
    sudo setenforce 0
sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config
  • Disable swap space
    sudo sed -i '/swap/d' /etc/fstab
sudo swapoff -a
  • Configure Firewall rules
    sudo firewall-cmd --permanent --add-port=6443/tcp
sudo firewall-cmd --permanent --add-port=2379-2380/tcp
sudo firewall-cmd --permanent --add-port=10250/tcp
sudo firewall-cmd --permanent --add-port=10251/tcp
sudo firewall-cmd --permanent --add-port=10252/tcp
sudo firewall-cmd --permanent --add-port=10255/tcp
sudo firewall-cmd --reload
  • Install necessary Kubernetes tools (kubelet, kubectl, kubeadm)
    sudo yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
sudo systemctl enable --now kubelet
  • Initialize Master Node
    sudo kubeadm init --pod-network-cidr=10.244.0.0/16

Incase it errors out run the below commands: 

sudo rm /etc/containerd/config.toml
sudo systemctl restart containerd

Note

Save the kubeadm join tokenThis will be used in the last step of the worker node where it will be used to join with k8s master node.Ex: kubeadm join < clustet IP>:6443 –token < > –discovery-token-ca-cert-hash < sha256:**** >

  • bashsudo mkdir -p $HOME/.kube

sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config * Install flannel networking driver

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
  • Configure networking requirements
    cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF

    cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF

    sudo sysctl --system

    sysctl net.bridge.bridge-nf-call-iptables net.bridge.bridge-nf-call-ip6tables net.ipv4.ip_forward
  • If pods are not stable apply the below configuration
    sudo mkdir -p /etc/containerd/

    containerd config default | sudo tee /etc/containerd/config.toml

    sudo sed -i 's/SystemdCgroup \= false/SystemdCgroup \= true/g' /etc/containerd/config.toml

Installation Steps for Worker Nodes

Now, Perform the below steps on all the worker nodes one by one.

Step 1: Create 4px user

  • Create 4px user

sudo useradd 4px
sudo passwd 4px

  • Add 4px user to the sudoers file

sudo visudo
4px ALL=(ALL) NOPASSWD: ALL #Add this line to sudoers file

echo "$(whoami) ALL=(4px) NOPASSWD: ALL" | sudo tee -a /etc/sudoers.d/4px

  • Switch to 4px user

sudo su - 4px

Step 2: Install Docker

  • Configure the yum repo to install the docker
    sudo yum install -y yum-utils
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
  • Install and configure the docker
    sudo yum install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo usermod -aG docker $USER
newgrp docker
  • Start and enable the docker service
    sudo systemctl start docker
sudo systemctl enable docker.service
sudo systemctl enable containerd.service

Step 3: Install Kubernetes

  • Configure the yum repo to install Kubernetes
    cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/v1.29/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/v1.29/rpm/repodata/repomd.xml.key
exclude=kubelet kubeadm kubectl cri-tools kubernetes-cni
EOF
  • Disable SELinux
    sudo setenforce 0
sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config
  • Disable swap space
    sudo sed -i '/swap/d' /etc/fstab
sudo swapoff -a
  • Configure Firewall rules
    sudo firewall-cmd --permanent --add-port=6443/tcp
sudo firewall-cmd --permanent --add-port=2379-2380/tcp
sudo firewall-cmd --permanent --add-port=10250/tcp
sudo firewall-cmd --permanent --add-port=10251/tcp
sudo firewall-cmd --permanent --add-port=10252/tcp
sudo firewall-cmd --permanent --add-port=10255/tcp
sudo firewall-cmd --reload
  • Install necessary Kubernetes tools (kubelet, kubectl, kubeadm)
    sudo yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
sudo systemctl enable --now kubelet
  • Configure networking requirements
    cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF

    cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF

    sudo sysctl --system

    sysctl net.bridge.bridge-nf-call-iptables net.bridge.bridge-nf-call-ip6tables net.ipv4.ip_forward
  • Run the join command which you copied after the initialization of the master node.
    sudo kubeadm join < clustet IP>:6443 --token < > \ --discovery-token-ca-cert-hash < sha256:******** >
  • Now, Login to the master node run the get nodes command, and check if all nodes are in ready state or not.
    kubectl get nodes

    [demo@rnd-1 4px]$ kubectl get nodes
NAME    STATUS   ROLES           AGE   VERSION
rnd-1   Ready    control-plane   35d   v1.28.2
rnd-2   Ready    <none>          35d   v1.28.2
rnd-3   Ready    <none>          35d   v1.28.2
rnd-4   Ready    <none>          35d   v1.28.2

(Optional) Enable Scheduling on Master Node

In a two-node setup, you may want to enable scheduling on the master node. Since this is a non-standard approach, you must run the following command to enable it.

  1. kubectl get nodes

  2. kubectl taint nodes <master-node-name> node-role.kubernetes.io/control-plane:NoSchedule-

  3. Verify:
    kubectl describe node <master-node-name> | grep Taint

Resource Limit Adjustments

If the VMs are created with just 2 vCPUs and 16GB RAM (like the current demo environment), remove resource limits/requests from deployments to avoid scheduling issues. After deployment, scale down the setup deployment to prevent it from resetting the resource limits.

References:

https://phoenixnap.com/kb/how-to-install-kubernetes-on-centos

https://phoenixnap.com/kb/install-kubernetes-on-ubuntu

https://www.linuxtechi.com/how-to-install-kubernetes-cluster-rhel/