Minikube - Persistence Volume

One of the important concepts in Kubernetes is to define persistence volumes to retain the data when nodes get deleted or restarted. I have not studied the methods for persistent volume in detail, but to fulfill my requirements, I found choosing persistent volume with NFS(network file system) to be very useful, so that I can assign an external volume mounted as NFS as a persistent volume in Kubernetes' pods. 

Some of the mechanisms of the persistent volume are 

1) Hostpath => Volume in the host system of the node. It is destroyed when a node is restarted or removed. 

2) Local => accessible to pods in  a node (any mounted partition can be assigned)

3) NFS => This one is the best for me. I can use any system in the network and mount the network file system (NFS) to use as a persistent volume. 

NFS Persistent Volume 

 https://kubernetes.io/docs/concepts/storage/volumes/#local 

In summary, we need to install nfs-kernel-server on the host system.

sudo apt update

sudo apt install nsf-kernel-server

Then we create a shared directory

sudo mkdir /Backup/k8s-volume

cd /Backup/k8s-volume

NFS will translate any root operations on the client to the nobody:nogroup credentials as a security measure. Therefore, you need to change the directory ownership to match those credentials.

sudo chown nobody:nogroup /Backup/k8s-volume

sudo service nfs-kernel-server restart

We now want to config NFS exports on the host

The following line in /etc/exports will suffice 

/Backup/k8s_volume *(rw,sync,no_subtree_check,no_root_squash,no_all_squash,insecure)

The exported volume can be mounted anywhere from the network. So even if the nodes got destroyed or restarted, we have persistent data safe in the network system. 

sudo exportfs -rav  (exports all filesystem paths)

sudo exportfs -v  (verify)

Finally, restart the server:

sudo systemctl restart nfs-kernel-server

Now the server is running. If the server is using some firewall techniques, we have to allow the NFS port 2049. 

Command to verify (from client)

sudo mount -t nfs <server(ip/hostname)>:/Backup/k8s_volume /mnt

The files will be mounted into /mnt folder 

To unmount

sudo umount /mnt

Note: There is much other information in the link given above. For Kubernetes, this will be enough.

Now let's create pv.yaml file with NFS persistent volume:

apiVersion: v1

kind: PersistentVolume

metadata:

name: persistent-volume

labels:

type: nfs

app: k8s_volume

spec:

capacity:

storage: 10Gi

volumeMode: Filesystem

accessModes:

- ReadWriteMany

storageClassName: manual

#hostPath:

# path: /Backup/temp/volumes

nfs:

path: /Backup/k8s_volume

server: 192.168.x.xxx

readOnly: false

kubectl apply -f pv.yaml

kubectl get pv 

The result shows the create persistent volume. We can then allocate the volume needed for pods, which is called "persistent volume claim" or pvc. For example: pvc.yaml

apiVersion: v1

kind: PersistentVolumeClaim

metadata:

name: postgres-volume-claim

labels:

app: k8s_volume

spec:

storageClassName: manual

accessModes:

- ReadWriteMany

resources:

requests:

storage: 1Gi  

Note the app name in pv.yaml and pvc.yaml is the same so that PVC knows from where the volume to claim is. 

Preparing Mini-Server - Part 2 (Minikube)

Minikube

While docker and docker-compose are simple and fulfill the basic requirement, I prefer Kubernetes' way of managing containers also can be scaled. So, if there is some problem in one pod, Kubernetes automatically fixes the pod, recovering the downtime. 

Kubernetes can be best managed by creating multiple nodes, but setting Kubernetes cluster at home seems advanced. So, we need to find out a way to set up a Kubernetes cluster in one machine in a single node, and Minikube is developed for that. There are some alternatives to Minikube like Kind and k3s, but I am going to install Minikube and learn Kubernetes concepts. My final goal will be to fully deploy my services into Kubernetes, the services are currently running on Docker. 

So, here I describe all the processes to make Minikube's single node up and running. 

1) Installation of docker (in Ubuntu, tested in 22.04)

sudo apt-get remove docker docker-engine docker.io containerd runc

sudo mkdir -m 0755 -p /etc/apt/keyrings

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

echo   "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \

 $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Test the installation

sudo docker run hello-world

Check if everything is fine

sudo service docker status

sudo systemctl status docker.service  

sudo systemctl is-enabled docker.service 

sudo systemctl is-active docker.service 

sudo docker compose version

Add user to docker group (so that you do not need to provide sudo with docker) 

Check if the docker group exists in /etc/group file. If no, add a docker group. 

sudo groupadd docker

Add the user to the docker group.

sudo usermod -aG docker $USER

Restart the system, test 

docker run hello-world

Refefence: https://docs.docker.com/engine/install/ubuntu/ 

2) Installation of Minikube

wget https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64

chmod +x minikube-linux-amd64 

sudo install minikube-linux-amd64 /usr/local/bin/minikube

Verify

minikube version

Start

minikube start

Ref: https://r2schools.com/how-to-install-minikube-on-ubuntu-22-04-lts/

3) Installation of kubectl

curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl

chmod +x kubectl 

sudo install kubectl /usr/local/bin/kubectl

Verify

kubectl version -o json --client 

Ref: https://r2schools.com/how-to-install-minikube-on-ubuntu-22-04-lts/

4) Enable autocomplete for kubectl

• Check if bash-completion is already installed. 

    type _init_completion

          

        If already installed, you will see some messages. If not installed, install it.

       apt-get install bash-completion 

• Enable kubectl autocompletion and reload bash

        kubectl completion bash | sudo tee /etc/bash_completion.d/kubectl > /dev/null

• Setup alias for kubectl

    echo 'alias k=kubectl' >>~/.bashrc

• Enable the alias for auto-completion and reload bash. (source ~/.bashrc)

    echo 'complete -o default -F __start_kubectl k' >>~/.bashrc

  Now, autocomplete will work for the alias too :)

Reference: https://spacelift.io/blog/kubectl-auto-completion 

Preparing Mini-Server - Part 1 (System)

BACKGROUND

In this article, I am going to share my experience with mini-server preparation. Actually, this whole job I have carried out as my hobby project and a kind of research & development. In my free time, I develop interesting projects and want to run them somewhere with 100% availability.

I got one free web space from somewhere (don't remember now) and I could run my small PHP project to save my personal passwords. Actually, I did not have much interest and wanted to switch to java which I am comfortable with. I then started looking for some java (free) servers where I can host my applications, unfortunately, I did not find any such provider. I had one notebook which I made run for 24 hours as a server and deployed my java applications. One big problem with this is it used to consume lots of power (almost 28W) and to run for 24 hours is, I think, not a good idea. Also, the noise produced by the notebook was also noticeable(when it runs for 24 hours).

Then I decided to purchase a virtual private server from Amazon. I was very excited to have almost 30 GB of space and 1 GB of RAM. I could run simple programs without any problem. More interesting is that I have full access to the system, and is very secure to save my private data there, than some random webspace provider. I had to pay ca. 5 EURO per month for this amazing service provided by amazon (EC2). And as compared to the electricity cost of running a full system, it was economical too. Later I found an economical option called Vultr(https://vultr.com), which I used for a couple of months, and later unsubscribed because of inactivity. IAAS provided by Amazon and Vultr was amazing, the only problem is resources that 1GB of RAM and 30 GB of SSD were not enough to run many applications parallel. You have to upgrade the resources costing more money. There were options too, to only run specific time intervals which sacrificed the availability of the service.

MINI-PC

The idea of MINI-PC came when I realize that instead of letting your data online in the cloud (which in my opinion has some sort of security and availability risks), you can have your data on your server and make it available on the internet through your router. So, you have full access to your system and can manage it in your own way. Another plus point you can get high SSD volume and more RAM. Your data will be stored locally, it does not go external world, by some security mechanism, you can fully secure your sensitive data. 

Factors to consider for a mini-server

We have to consider some points before you purchase a PC for a server to let it run for 24 hours. There are many min-pc available in the market. There are powerful mini-pc that consume more power and run warmer. The cooling system should run to make the system cool. Although they can do complex tasks faster, for a server that runs 24 hours a day, we want it to be as quiet as possible. Mini servers normally run without any graphical display, and they are not meant for gaming. So, a normal graphics card that comes with a processor should be enough. CPUs should consume as low power as possible (preferably 10W-15W - Celeron) and with this power bound as many cores as possible.  

Regarding fans, some mini-pcs have louder fans installed which can be heard in the room. For a server running 24 hours, PCs with louder fans are NOT recommended. There are BIOS settings that can set the fan speed based on temperature, to make the fan run quieter. For example, when I turned on the new mini-PC from NUC, the fan was always running, because the default setting was to run cooler with the fan always running. After changes in BIOS, the fan stopped running, and it was fully quiet. The minimum temperature to run the fan was made increased, so that fan starts running when the CPU temperature reaches that minimum temperature. 

I have done tons of research, and even though the price is higher compared to others, I could not compromise the build quality of Intel NUC PCs. My expectation was fulfilled by Intel NUC BOXNUC6CAYH (4 core intel Celeron & 8 GB memory), which is running since 2019 (almost 4 years now!) without any issues. The good part is that it has a passive fan and runs very cool 24 hours a day. I reboot the system very when there are kernel updates, that's it!

OPERATING SYSTEM

I have NOT done any research regarding the best operating system for the mini-pc. I needed lightweight, without any bloatware, and ubuntu-server met the requirements.

 The installation of the minimal server is very light and you can install the required packages/services later. After installation, I installed open-ssh to access the system remotely. Surprisingly, in contrast to windows, the memory usage was just 400 megabytes and CPU usage was almost zero. 

Enable Remote Login

To enable remote login, we have to install the open-ssh service. To allow remote login with password, you have to uncomment the following line in /etc/ssh/sshd_config file.

# PasswordAuthentication yes

Configure Network

One of the challenging tasks after installation of the ubuntu-server is to configure the network and assign an IP address to the system. 

>> Command to check all the interfaces and IP address

     ip a 

    We can see the network information including the IP addresses of all the interfaces.

Here, the first one is the lookback interface. The second one is ethernet and the third one is a wireless interface. The default network configuration is DHCP, so the IP address is assigned by a DHCP server in the network. If we want to assign static IP address, then we have to assign IP address statically. For that, we need to edit the file in the /etc/netplan folder. There are yaml files and we change dhcp to static and provided our information manually.

Wifi:

A typical wifi configuration looks something like this: 

Ethernet:

A typical ethernet configuration looks something like this:

Here I have disabled DHCP, so I have provided the Ip address, nameservers, and gateway myself. 

Note: to avoid the network verification time (ca. 2 minutes) while booting, we have made optional true, which skips the network checking.

After you have changes in one of those files, you run the following command to verify:

sudo netplan generate 

sudo netplan apply 

If you see no error messages there, you are good to go, otherwise, you have to fix the configuration problems. 

Notes:

1) Please don't enable disk encryption, which needs human intervention to provide the passphrase when the system is rebooted(which is infeasible to do remotely)

2) Use this command if "lsblk" or "df -h" not showing full disk size

sudo lvextend -l +100%FREE /dev/mapper/ubuntu--vg-ubuntu--lv