Automate K3s Cluster Installation on Flatcar Container Linux
This tutorial is the fourth and the last part of a series on running Flatcar Container Linux. (Part 1), (Part 2). (Part 3).
In this tutorial, we install a highly available K3s cluster along with Flatcar Container Linux. K3s is a stripped-down Kubernetes distribution designed for edge computing deployments.
To complete this tutorial, you need to be familiar with the installation process of putting Flatcar Linux on remote bare metal servers. We extend the scenario to automate the installation of K3s along with the OS.
Leveraging Ignition to Configure K3s Installation
Let’s do a quick recap of the bare metal installation process of Flatcar Container Linux.
We take the help of two Ignition files to configure the OS. Ignition is a provisioning utility designed specifically for container OSs like Flatcar Container. The first file runs during the PXE boot installation process that ends by downloading and persisting the second Ignition file to the disk.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
systemd: units: - name: installer.service enabled: true contents: | [Unit] Requires=network-online.target After=network-online.target [Service] Type=forking TimeoutStartSec=600 RemainAfterExit=yes ExecStart=/usr/bin/sh -c "flatcar-install -d /dev/sda -i /opt/ignition.json && udevadm settle && systemctl reboot" [Install] WantedBy=multi-user.target storage: files: - path: /opt/ignition.json filesystem: root mode: 777 contents: remote: url: http://10.0.0.5:8080/node-1-ignite-boot.ign |
The persisted Ignition file, node-1-ignite-boot.ign stored as /opt/ignition.yaml is used during the boot time to configure the OS. It configures users, hostname, NTP servers, and other settings.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
passwd: users: - name: core ssh_authorized_keys: - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDGdByTgSVHq... storage: files: - path: /etc/hostname filesystem: root mode: 0644 contents: inline: node-1 |
We can also include systemd unit files in this file that may run during the boot process. We will leverage this to install K3s.
Installing K3s Server on the First Node
The typical installation of K3s includes running the below command which first downloads a script and then pipes the output to the shell.
|
1 |
curl -sfL https://get.k3s.io | sh - |
We will turn this step into a systemd unit file that will run at the boot time. To avoid reinstalling K3s, we will check the presence of /opt/bin/k3s, and skip the process if it’s available.
On the gateway server, modify the node-1-ignite-boot.yaml file to add K3s installation process.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 |
systemd: units: - name: k3s-install.service enabled: true contents: | [Unit] Description=Run K3s script Wants = network-online.target After = network.target network-online.target ConditionPathExists=/opt/k3s-install.sh ConditionPathExists=!/opt/bin/k3s [Service] Type=forking TimeoutStartSec=180 RemainAfterExit=yes KillMode=process Environment="K3S_TOKEN=secret_edgecluster_token" Environment="INSTALL_K3S_EXEC=--cluster-init" ExecStart=/usr/bin/sh -c "/opt/k3s-install.sh" [Install] WantedBy=multi-user.target passwd: users: - name: core ssh_authorized_keys: - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDGdByTgSVHq... storage: files: - path: /etc/hostname filesystem: root mode: 0644 contents: inline: node-1 - path: /opt/k3s-install.sh filesystem: root mode: 777 contents: remote: url: https://get.k3s.io |
First, we download the K3s installation script from https://get.k3s.io and save it to /opt/k3s-install.sh.
The below snippet under the storage section will perform this step.
|
1 2 3 4 5 6 |
- path: /opt/k3s-install.sh filesystem: root mode: 777 contents: remote: url: https://get.k3s.io |
We then create a systemd unit file that executes after the network service is started. It checks for /opt/bin/k3s file and runs the downloaded installation script only if the file is not found.
Note that we are passing the parameters expected by the K3s script through the environment variables.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
systemd: units: - name: k3s-install.service enabled: true contents: | [Unit] Description=Run K3s script Wants = network-online.target After = network.target network-online.target ConditionPathExists=/opt/k3s-install.sh ConditionPathExists=!/opt/bin/k3s [Service] Type=forking TimeoutStartSec=180 RemainAfterExit=yes KillMode=process Environment="K3S_TOKEN=secret_edgecluster_token" Environment="INSTALL_K3S_EXEC=--cluster-init" ExecStart=/usr/bin/sh -c "/opt/k3s-install.sh" [Install] WantedBy=multi-user.target |
Convert the YAML file to JSON by running the configuration transpiler.
|
1 |
ct < node-1-ignite-boot.yaml > node-1-ignite-boot.ign |
Continue with the rest of the PXE boot process to install Flatcar Linux followed by K3s.
Automating K3s Server Installation on Remaining Nodes
The entire installation process remains the same except for generating the boot time Ignition files.
We will need to include the K3s configuration for joining the nodes to the first server.
The below Ignition file is used for configuring the second node:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
systemd: units: - name: k3s-install.service enabled: true contents: | [Unit] Description=Run K3s script Wants = network-online.target After = network.target network-online.target ConditionPathExists=/opt/k3s-install.sh ConditionPathExists=!/opt/bin/k3s [Service] Type=forking TimeoutStartSec=180 RemainAfterExit=yes KillMode=process Environment="K3S_TOKEN=secret_edgecluster_token" Environment="INSTALL_K3S_EXEC='--server' 'https://10.0.0.70:6443'" ExecStart=/usr/bin/sh -c "/opt/k3s-install.sh" [Install] WantedBy=multi-user.target passwd: users: - name: core ssh_authorized_keys: ssh_authorized_keys: - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDGdByTgSVHq... storage: files: - path: /etc/hostname filesystem: root mode: 0644 contents: inline: node-2 - path: /opt/k3s-install.sh filesystem: root mode: 777 contents: remote: url: https://get.k3s.io |
Notice the environment variables used for configuring the K3s node.
|
1 2 |
Environment="K3S_TOKEN=secret_edgecluster_token" Environment="INSTALL_K3S_EXEC='--server' 'https://10.0.0.70:6443'" |
Except for the hostname, the same configuration is applicable to the third node of the cluster.
Finalizing and Verifying the Configuration
At this point, you should have 6 Ignition files in the /var/lib/tftp/ignition – three files PXE booting each node and three files for configuring and installing K3s on each node.
In the /var/lib/tftp/pxelinux.cfg directory, make sure that you have the three PXE boot files for each node and a link mapped to the IP address of the nodes. Refer to the previous part of this tutorial for the details on this topic.
Ensure that the PXE boot server is running, and reboot the nodes. Within a few minutes, you will have a fully configured, highly available K3s cluster deployed on bare metal servers running Flatcar Container Linux.
Janakiram MSV’s Webinar series, “Machine Intelligence and Modern Infrastructure (MI2)” offers informative and insightful sessions covering cutting-edge technologies. Sign up for the upcoming MI2 webinar at http://mi2.live.
Feature image by Pascal van de Vendel on Unsplash.