What news from AWS re:Invent last week will have the most impact on you?
Amazon Q, an AI chatbot for explaining how AWS works.
Super-fast S3 Express storage.
New Graviton 4 processor instances.
Emily Freeman leaving AWS.
I don't use AWS, so none of this will affect me.
AI / Edge Computing / Serverless / Tech Life

Implement IoT Predictive Maintenance with PubNub and Azure Machine Learning

Jan 19th, 2018 6:00am by
Featued image for: Implement IoT Predictive Maintenance with PubNub and Azure Machine Learning

One of the key benefits of an industrial Internet of Things-based system is predictive maintenance. Based on the metrics ingested by the connected devices, a machine learning algorithm detects anomalies to predict potential failures. This not only saves a few million dollars for enterprises but it also ensures minimal downtime involved in dealing with defective equipment.

This tutorial series will help developers apply predictive maintenance to devices connected to the PubNub Data Stream Network (DSN), a global third-party streaming network service. It will explore the concepts of the recently announced PubNub support for MQTT, along with Functions, the serverless environment of PubNub. It will highlight how to integrate with third-party cloud services such as Microsoft Azure.

The scenario for this tutorial involves using a simulated turbine that ingests key metrics such as the fan speed, vibrations, noise levels, and temperature. Each metric ingested into PubNub DSN will be sent to Azure ML to predict possible anomalies. A PubNub Function invokes the REST endpoint exposed by an Azure ML model that’s trained to predict the anomalies. When the Web Service repeatedly reports anomalies, the PubNub Function sends a message to a simulated alarm. If the alarm goes unattended for a predefined amount of time, it automatically shuts down the turbine in question.


PubNub, the real-time data streaming network company was started in 2010 by Stephen Blum and Todd Greene with a vision to develop a streaming network for developers to build real-time apps as easily as building a web page. The PubNub Data Streaming Network (DSN) is currently available in more than 15 Points of Presence (PoP) across the world enabling developers to build connected applications and services.

I explored PubNub when I wanted to connect remote IoT devices with other devices. What instantly hit me was the fact that I could connect two Raspberry Pi devices with no server running in the cloud. With less than ten lines of code, I could easily connect both ends. There are more than 70 SDKs from PubNub covering the popular languages and platforms including JavaScript, .NET, Java, Python, and Swift. Any developer can get up and running with PubNub messaging in no time.

To appreciate the power of PubNub, think of it as a Content Delivery Network (CDN) but for streams. PubNub is to streams what Akamai is to object storage. Each message sent to PubNub DSN will be delivered across 15 locations covering Asia, Europe, and Americas. Backed by 99.999 percent SLA, these messages are guaranteed to be delivered within 250ms anywhere in the world.

PubNub has added the ability to intercept and dynamically manipulate the messages in the stream. This means developers can take control of messages on the fly to change the content and target different delivery channels. This programmability of streams is delivered via the serverless offering called PubNub Functions. Like AWS Lambda and Azure Functions, developers can write JavaScript code snippets that are invoked each time a message is sent. This opens up many opportunities for inspecting the data and performing custom actions.

There are three key players in the tutorial we are going to explore — devices, PubNub DSN, and Azure ML. Apart from orchestrating the message flow between the devices layers consisting of a turbine and alarm, PubNub is responsible for invoking the ML Web Service hosted in Azure. The heavy lifting involved in training and deploying the machine learning model is delegated to Azure ML.

To run this sample, you need an account with and Azure ML Studio.

The Devices Layer

We have two entities — a turbine and alarm — in the device layer. Both of them are connected to the PubNub DSN.

The turbine uses standard MQTT libraries to publish messages to PubNub. Recently, PubNub added native support for MQTT, which enables developers to publish messages to the network using familiar MQTT libraries such as Paho. The MQTT endpoint is exposed at The authentication mechanism for the MQTT endpoint is based on the combination of publisher and subscriber keys available at The client id of the MQTT client uses these keys to construct a unique identifier. The clientid-based authentication ensures that there are no changes made to the standard MQTT program. This is very helpful in connecting existing devices that are using MQTT to the PubNub network. Refer to the MQTT blog post and MQTT PubNub Bridge documentation for more details on PubNub’s support for MQTT.

The code snippet below represents a turbine sending messages to PubNub via MQTT.

Install the modules and dependencies before running the script:

Firstly, the code is based on Paho library that’s popular among developers. The broker is pointing to, which is the endpoint of the hosted MQTT bridge. Notice the way the clientid is constructed. It’s using the combination of the publisher and subscriber keys along with a client identifier. This is essential for the clients to get authenticated with the PubNub platform.

In normal conditions, the telemetry data generated by the Python code falls under acceptable levels. We can change the metrics when we need to simulate an anomaly by turning the variable fault to True.

If you are new to MQTT, refer to the deep dive tutorial that I published at The New Stack.

The code publishes the telemetry data to the MQTT topic turbine/messages while subscribing to the turbine/command. When an alarm device wants to shut down the faulty turbine, it publishes the command on the same topic.

Interestingly, the MQTT topics, turbine/message and turbine/command map to the channels of PubNub, which follow the naming convention of turbine.message and turbine.command. Any PubNub client can retrieve the telemetry data or send a command to the turbine through these channels. This interoperability makes it easy to integrate MQTT clients with PubNub native clients.

Native PubNub clients can also subscribe to the topics using the wildcard notation. For example, subscribing to turbine.* topic would enable the client to receive all the messages sent to the parent, turbine.

In the next part of this tutorial series, we will build the native PubNub client in Python. We will also implement serverless computing through PubNub Functions to invoke the predictive maintenance machine learning model in Azure ML. Stay tuned!

Microsoft is a sponsor of The New Stack.

Feature image via Pixabay.

Group Created with Sketch.
TNS owner Insight Partners is an investor in: The New Stack.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.