Edge / IoT / Kubernetes / Machine Learning

Tutorial: Configure, Deploy an Edge Application on Cloud Native Edge Infrastructure

5 Oct 2020 1:55pm, by

This is the last installment of the 4-part series (Part 1) (Part 2) (Part 3) where we configure and deploy an artificial intelligence workload running on an “edge” Internet of Things (AIoT) workload on cloud native edge infrastructure. The application does predictive maintenance of turbines. For the background and explanation of the use case, refer to the previous part of the tutorial.

Start by cloning the GitHub repository that contains the code, configuration, and Kubernetes deployment manifests. Feel free to explore the source code, configuration, and manifests available in the repository. You can build container images for each of the services to store them in a private registry.

As we start deploying each microservice, I will discuss the design decisions and configuration choices.  If you have a K3s cluster configured with Project Calico and Portworx, you can deploy the entire application without building the images.

If you want to deploy one artifact at a time, start with the namespace. The workload runs in a dedicated namespace, aiot.

Deploying the Mosquitto MQTT Broker

The Mosquitto pod acting as the MQTT broker relies on a persistent volume claim to persist logs and data. This PVC uses dynamic provisioning based on a Portworx storage class which is optimized for databases. The same storage class is used for the InfluxDB time-series database.

Deploy the storage class followed by the Mosquitto service.

Deploying Fan Simulators

The fan simulators are configured as pods with environment variables pointing to the MQTT broker and the topic associated with the telemetry. The environment variable FAULT will decide if the simulator publishes anomalous data. The DEVICE_ID variable assigns an arbitrary value to the device identifier.

The InitContainer will wait for 60 seconds for the Mosquitto broker to become available before timing out. This will ensure that the pod doesn’t experience CrashLoopBackOff while waiting for the dependent service.

Deploy two fans one of which is configured to ingest anomalous data.

Now, we have two simulators publishing telemetry to an MQTT topic.

Deploying AI Inference and Prediction Service

The inference pod downloads the latest version of the model and exposes that as a REST endpoint. An InitContainer checks for the presence of the model in a well-known directory and pulls the compressed TensorFlow model only if it’s missing. This approach avoids downloading and copying the model each time the deployment is scaled.

As the inference service scales, the model needs to be available to multiple pods through a shared filesystem. To enable this, we define a different Portworx storage class with the SharedV4 flag to create a globally shared namespace volume that can be used by multiple pods.

After creating the storage class, let’s create the inference service.

With the inference service in place, we can deploy the predictor microservice that acts as the intermediary between the MQTT broker and the AI model.

The values related to the MQTT broker and the endpoint of the inference service are passed as environment variables.

Since the inference microservice is using shared volume, we can easily scale the number of replicas.

Deploying Telegraf and InfluxDB

InfluxDB is used as a time-series database to store the telemetry data coming from the fan simulators and the prediction service. It is configured as a stateful set backed by the PVCs created from the same Portworx storage class used by Mosquitto.

Telegraf connects InfluxDB with Mosquitto through a configuration file which is created as a Kubernetes config map.

Check the logs of the Telegraf pod to confirm the flow of messages from Mosquitto to InfluxDB.

Deploying Grafana and Configuring the Dashboard

A config map associated with Grafana pod configures InfluxDB as the datasource. This bundling helps us quickly import an existing dashboard.

Deploy Grafana with the below command:

After accessing the Grafana dashboard, login with admin/admin and change the password. Go to the manage dashboards section and click on the import button.

Copy the content of fan.json from the dashboard directory, paste it into the textbox, and click the load button.

With the configuration in place, you would be able to access the below dashboard:

Configuring Network Policies

Since the K3s cluster is configured with Calico-based CNI, we can secure the network. A sample policy to prevent access to the AI inference is included in the deploy directory.

Janakiram MSV’s Webinar series, “Machine Intelligence and Modern Infrastructure (MI2)” offers informative and insightful sessions covering cutting-edge technologies. Sign up for the upcoming MI2 webinar at http://mi2.live.

Feature image by Dmitrii Bardadim from Pixabay.

A newsletter digest of the week’s most important stories & analyses.