Tutorial: Configure, Deploy an Edge Application on Cloud Native Edge Infrastructure
This is the last installment of the 4-part series (Part 1) (Part 2) (Part 3) where we configure and deploy an artificial intelligence workload running on an “edge” Internet of Things (AIoT) workload on cloud native edge infrastructure. The application does predictive maintenance of turbines. For the background and explanation of the use case, refer to the previous part of the tutorial.
Start by cloning the GitHub repository that contains the code, configuration, and Kubernetes deployment manifests. Feel free to explore the source code, configuration, and manifests available in the repository. You can build container images for each of the services to store them in a private registry.
As we start deploying each microservice, I will discuss the design decisions and configuration choices. If you have a K3s cluster configured with Project Calico and Portworx, you can deploy the entire application without building the images.
|
1 |
kubectl apply -f deploy/ |
If you want to deploy one artifact at a time, start with the namespace. The workload runs in a dedicated namespace, aiot.
|
1 |
kubectl apply -f deploy/aiot-ns.yaml |
Deploying the Mosquitto MQTT Broker
The Mosquitto pod acting as the MQTT broker relies on a persistent volume claim to persist logs and data. This PVC uses dynamic provisioning based on a Portworx storage class which is optimized for databases. The same storage class is used for the InfluxDB time-series database.
|
1 2 3 4 5 6 7 8 |
kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: db-sc provisioner: pxd.portworx.com parameters: io_profile: "db_remote" repl: "3" |
Deploy the storage class followed by the Mosquitto service.
|
1 |
kubectl apply -f deploy/db-sc.yaml |
|
1 |
kubectl apply -f deploy/mosquitto.yaml |
Deploying Fan Simulators
The fan simulators are configured as pods with environment variables pointing to the MQTT broker and the topic associated with the telemetry. The environment variable FAULT will decide if the simulator publishes anomalous data. The DEVICE_ID variable assigns an arbitrary value to the device identifier.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
initContainers: - name: wait-for-mosquitto image: janakiramm/wait args: ["--timeout=60", "mosquitto:1883"] containers: - name: fan-1 image: janakiramm/fan imagePullPolicy: Always env: - name: MQTT_HOST value: "mosquitto" - name: MQTT_TOPIC value: "fan/messages" - name: FAULT value: "0" - name: DEVICE_ID value: "1" |
The InitContainer will wait for 60 seconds for the Mosquitto broker to become available before timing out. This will ensure that the pod doesn’t experience CrashLoopBackOff while waiting for the dependent service.
Deploy two fans one of which is configured to ingest anomalous data.
|
1 |
kubectl apply -f deploy/fan-1.yaml |
|
1 |
kubectl apply -f deploy/fan-2.yaml |
Now, we have two simulators publishing telemetry to an MQTT topic.
Deploying AI Inference and Prediction Service
The inference pod downloads the latest version of the model and exposes that as a REST endpoint. An InitContainer checks for the presence of the model in a well-known directory and pulls the compressed TensorFlow model only if it’s missing. This approach avoids downloading and copying the model each time the deployment is scaled.
As the inference service scales, the model needs to be available to multiple pods through a shared filesystem. To enable this, we define a different Portworx storage class with the SharedV4 flag to create a globally shared namespace volume that can be used by multiple pods.
|
1 2 3 4 5 6 7 8 |
kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: infer-sc provisioner: pxd.portworx.com parameters: repl: "1" sharedv4: "true" |
|
1 |
kubectl apply -f deploy/infer-sc.yaml |
After creating the storage class, let’s create the inference service.
|
1 |
kubectl apply -f deploy/infer.yaml |
With the inference service in place, we can deploy the predictor microservice that acts as the intermediary between the MQTT broker and the AI model.
The values related to the MQTT broker and the endpoint of the inference service are passed as environment variables.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
containers: - name: predict image: janakiramm/predict env: - name: MQTT_HOST value: "mosquitto" - name: MQTT_DEV_TOPIC value: "fan/messages" - name: MQTT_PREDICT_TOPIC value: "fan/anomaly" - name: SCORING_URL value: "http://infer:5000/score" |
|
1 |
kubectl apply -f deploy/predict.yaml |
Since the inference microservice is using shared volume, we can easily scale the number of replicas.
|
1 |
kubectl scale deploy/infer --replicas=3 |
Deploying Telegraf and InfluxDB
InfluxDB is used as a time-series database to store the telemetry data coming from the fan simulators and the prediction service. It is configured as a stateful set backed by the PVCs created from the same Portworx storage class used by Mosquitto.
|
1 |
kubectl apply -f deploy/influxdb.yaml |
Telegraf connects InfluxDB with Mosquitto through a configuration file which is created as a Kubernetes config map.
|
1 |
kubectl apply -f deploy/telegraf.yaml |
Check the logs of the Telegraf pod to confirm the flow of messages from Mosquitto to InfluxDB.
|
1 |
TELEGRAF_POD=$(kubectl get pods -n aiot -l app=telegraf -o jsonpath='{.items[0].metadata.name}') |
|
1 |
kubectl logs -f $TELEGRAF_POD |
Deploying Grafana and Configuring the Dashboard
A config map associated with Grafana pod configures InfluxDB as the datasource. This bundling helps us quickly import an existing dashboard.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
apiVersion: v1 kind: ConfigMap metadata: name: grafana-datasources namespace: aiot data: influxdb.yaml: |- { "apiVersion": 1, "datasources": [ { "access":"proxy", "editable": true, "name": "influxdb", "orgId": 1, "type": "influxdb", "database": "fan", "url": "http://influxdb:8086", "version": 1 } ] } |
Deploy Grafana with the below command:
|
1 |
kubectl apply -f deploy/grafana.yaml |
After accessing the Grafana dashboard, login with admin/admin and change the password. Go to the manage dashboards section and click on the import button.
Copy the content of fan.json from the dashboard directory, paste it into the textbox, and click the load button.
With the configuration in place, you would be able to access the below dashboard:
Configuring Network Policies
Since the K3s cluster is configured with Calico-based CNI, we can secure the network. A sample policy to prevent access to the AI inference is included in the deploy directory.
|
1 |
kubectl apply -f deploy/netpol.yaml |
Janakiram MSV’s Webinar series, “Machine Intelligence and Modern Infrastructure (MI2)” offers informative and insightful sessions covering cutting-edge technologies. Sign up for the upcoming MI2 webinar at http://mi2.live.
Feature image by Dmitrii Bardadim from Pixabay.