How to set up your own Kubernetes Cluster?

Hi there, so I am working on a project that involves designing a Kubernetes cluster from the ground up. I believe this blog post is a good way to "microservice" my documentation so that everyone new to this can learn a tiny bit of Kubernetes - step by step!

What you will need

1. A minimum of two or more Virtual Machines / Machines where you want to set up your Kubernetes Cluster. I personally set up this cluster using 4 Ubuntu Virtual Machines. Details below.

   mukul@k8s-node1:~$ cat /etc/os-release
   NAME="Ubuntu"
   VERSION="18.04.6 LTS (Bionic Beaver)"
   ID=ubuntu
   ID_LIKE=debian
   PRETTY_NAME="Ubuntu 18.04.6 LTS"
   VERSION_ID="18.04"
   HOME_URL="https://www.ubuntu.com/"
   SUPPORT_URL="https://help.ubuntu.com/"
   BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
   PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
   VERSION_CODENAME=bionic
   UBUNTU_CODENAME=bionic
   mukul@k8s-node1:~$ lscpu
   Architecture:        x86_64
   CPU op-mode(s):      32-bit, 64-bit
   Byte Order:          Little Endian
   CPU(s):              4
   On-line CPU(s) list: 0-3
   Thread(s) per core:  1
   Core(s) per socket:  1
   Socket(s):           4
   NUMA node(s):        1
   Vendor ID:           GenuineIntel
   CPU family:          6
   Model:               85
   Model name:          Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz
   Stepping:            4
   CPU MHz:             2494.140
   BogoMIPS:            4988.28
   Hypervisor vendor:   VMware
   Virtualization type: full
   L1d cache:           32K
   L1i cache:           32K
   L2 cache:            1024K
   L3 cache:            39424K
   NUMA node0 CPU(s):   0-3
   Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon nopl xtopology tsc_reliable nonstop_tsc cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single pti ssbd ibrs ibpb stibp fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 invpcid rtm mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xsaves arat pku ospke flush_l1d arch_capabilities

   Note: You will need a Dual Core machine at the least to pass the pre-flight checks of Kubernetes.

2. You will need an internet connection to all these machines to download the required packages.

Getting Started

To get things started, this Kubernetes cluster will make user of systemd as the Cgroup driver, and docker as the container runtime. If you want more options, refer: https://kubernetes.io/docs/setup/production-environment/container-runtimes/

Perform these set of steps on all the machines:

• Update the apt package index:

  sudo apt update
  sudo apt install -y curl

• Install the latest validated version of Docker (validated by Kubernetes community)
  curl -fsSL https://get.docker.com | sudo VERSION=19.03 sh

!! Caution !! Note that this method of installation relies on providing root acces to a shell script obtained from the internet. Although the process is simple, use other installation methods for docker if this is a production system where security is important!

• Configure needed custom docker settings:

  sudo touch /etc/docker/daemon.json
  sudo chown $USER /etc/docker/daemon.json

  cat <<EOF | sudo tee /etc/docker/daemon.json
  {
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
  }
  EOF

  sudo mkdir -p /etc/systemd/system/docker.service.d 
  sudo chown root /etc/docker/daemon.json 
  sudo systemctl daemon-reload 
  sudo systemctl restart docker

• Add your local user to the docker group (after this command logout and back in to allow the current shell to realize the newly added group)
  sudo usermod -aG docker $USER

• Add Kubernetes official GPG key:
  curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

• Add Kubernetes repo to sources.list.d
  sudo apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main"

• Update the apt package index and install the kubelet,kubeadm and kubectl packages:
  sudo apt install -y kubelet=1.19.2-00 kubeadm=1.19.2-00 kubectl=1.19.2-00 kubernetes-cni=0.8.7-00

• Disable swap
  sudo swapoff -a

Perform these steps on the node that you decide as the master node

• On the master node:
  sudo kubeadm init --kubernetes-version=1.19.2 --pod-network-cidr=192.168.128.0/17

Note: You can provide any pod network here of your choice since we are going to install a weave-network overlay on the cluster that will make this configuration redundant. For more information on weave networks, check this out: https://www.weave.works/docs/net/latest/overview/

Note: Please take a note of the command which is the output of the above command, as that will used later for joining worker nodes with master. It will be of the format:
kubeadm join --token <token> <master-ip>:<master-port> --discovery-token-ca-cert-hash sha256:<hash>

• Configure kubectl to use the newly created admin credentials

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

• Verify that kubectl can connect to the newly created Kubernetes cluster
  kubectl get nodes

• Install the Weave network overlay:

  export KUBEVERBASE64=$(kubectl version | base64 | tr -d '\n')
  kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=${KUBEVERBASE64}"

• Check to see if the pods are running:
  kubectl get pods --all-namespaces
  The pods will start up over a short period of time.

Perform these steps on the other nodes that you want to join to the cluster

• Run the below command with sudo (which was the output of kubeadm init during master initialization)
  sudo kubeadm join --token <token> <master-ip>:<master-port> --discovery-token-ca-cert-hash sha256:<hash>
  A few seconds later, you should notice this node in the output from kubectl get nodes when run on the master node.

Note: In case, you missed to save the join command, you can get back by executing command "kubeadm token create --print-join-command" on k8s1 node.

• Watch the status of the K8s deployment (use CTRL-C to exit). Run this command on the master node.
  watch kubectl get nodes,svc,deploy,pods --all-namespaces

If you followed through till the end of this post, congratulations! You now have your own Kubernetes Cluster! ☺️