Tutorial: Build a Prometheus Dashboard for K3s with Wio Terminal
The Wio Terminal from Seeed Studio is a compact device with an Arduino-compatible microcontroller and a 2.4-inch LCD. It also boasts of an inbuilt WiFi and BLE radio for wireless connectivity.
The Wio Terminal had given me the idea to build a mini-Prometheus dashboard that shows the vital statistics of a Kubernetes cluster running at the edge. With a bit of effort, I translated the idea into a working prototype.
Though this tutorial works with any Kubernetes cluster, it is tested on a three-node K3s cluster running on the Odyssey Blue J4105 Mini PC from Seeed Studio.
Architecture
Prometheus exposes a REST API endpoint that can be queried to extract any available metric. The API responds with a JSON payload for any valid Prometheus query. We will leverage this API to extract the values of some of the key metrics.
Though Wio Terminal can use HTTP and JSON libraries to query and parse the responses, the size of the sketch may become overwhelming for the device. So, it’s better to move the heavy lifting to an external compute environment that sends a simplified, lightweight JSON with just the essential metrics. The wrapper that acts as the bridge/adapter between the Wio Terminal and Prometheus is deployed as a Kubernetes pod and exposed through NodePort/LoadBalancer service. Wio Terminal invokes this service periodically to get the required values.
Essentially, we have three components for this project — The Wio Terminal device, a wrapper service, and the Prometheus deployment running on Kubernetes.
The entire source code and configuration is available on GitHub.
Install and Configure Prometheus
In your Kubernetes cluster, install Prometheus through the Helm chart. This is the most efficient mechanism to deploy Prometheus and Grafana stack in your cluster.
Start by updating the Helm repository followed by the installation of the Chart in the Prometheus namespace.
|
1 2 3 |
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts helm repo add stable https://charts.helm.sh/stable helm repo update |
|
1 2 |
kubectl create ns prometheus helm install prometheus prometheus-community/kube-prometheus-stack --namespace prometheus |
Verify that the pods and services are installed correctly.
|
1 2 |
kubectl --namespace prometheus get pods -l "release=prometheus" kubectl --namespace prometheus get svc -l "release=prometheus" |
I converted the Prometheus service into a LoadBalancer by patching it with the below command:
|
1 |
kubectl -n prometheus patch svc prometheus-kube-prometheus-prometheus -p '{"spec": {"type": "LoadBalancer"}}' |
If you don’t have a LoadBalancer configured, change it to type NodePort.
Write the Prometheus Wrapper
The Prometheus wrapper service is a simple Flask web application that abstracts the queries and returns an aggregate response in JSON.
The below code snippet shows how the service wraps the query to get the number of nodes of the cluster.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
import flask from flask import json import requests import json app = flask.Flask(__name__) data={} PROMETHEUS = 'http://10.0.0.242:9090/api/v1/query' @app.route('/metrics') def metrics(): getMetrics() response = app.response_class( response=json.dumps(data), status=200, mimetype='application/json' ) return response def getMetrics(): global data response = requests.get(PROMETHEUS, params={'query': 'count(kube_node_info)'}) results = response.json() print("nodes:"+results['data']['result'][0]['value'][1]) data["nodes"]=results['data']['result'][0]['value'][1] if __name__ == "__main__": print("Loading server") app.run(host='0.0.0.0') |
We can add multiple queries to this service to expose additional metrics. For the complete code of the wrapper service, refer to this file on GitHub.
I turned the service into a Docker image and deployed it in the K3s cluster as a stateless deployment exposed as a service. The YAML specifications for these artifacts can be found in the deploy directory.
The below screenshots show the same:
Invoking the wrapper service endpoint returns the below JSON payload.
|
1 |
curl -s 10.0.0.243/metrics | jq |
Now that the service is deployed, let’s build the dashboard for the Wio Terminal.
Arduino Sketch for Wio Terminal
For building the dashboard, we will need to use the Arduino WiFi library, Arduino HTTP client, JSON library, and the TFT LCD library.
Before uploading the sketch to Wio Terminal, update the WiFi SSID and password in the arduino_secrets.h. Set the serverAddress[] and port to appropriate values.
Check the Serial Monitor of Arduino to ensure that the sketch is able to connect to the WiFi and the wrapper service endpoint.
You can now track the health of your K3s cluster remotely with this always-on mini dashboard.
Janakiram MSV’s Webinar series, “Machine Intelligence and Modern Infrastructure (MI2)” offers informative and insightful sessions covering cutting-edge technologies. Sign up for the upcoming MI2 webinar at http://mi2.live.