Off-The-Shelf Hacker: Lightweight Inter-Device Messaging with MQTT
In past Off-The-Shelf Hacker projects, like the PIR Yard Sensor, data was simply exchanged to/from the ESP8266 with my Linux notebook, using text, over the network. The methodology created one server (the notebook) and one client (the ESP8266 device).
While this worked fine for prototyping and getting to know Internet of Things basics, it doesn’t lend itself to deploying a constellation of sensors. What if you want a yard sensor at the southeast corner of the house and another pointing at the middle of the back yard? What if I go off the deep end and want to set up two dozen sensors to guard my humble little quarter-acre property from space aliens or other undesirables. Clearly, an established, standardized client-server setup is needed for integrating multiple sensors into a useful monitoring system.
Today, we’ll look at the MQTT messaging protocol for inter-device communications.
What Is MQTT?
MQTT is a mature client-server messaging system that uses a lightweight, publisher/subscriber model and runs over TCP/IP networks. It’s designed for connections between devices, in remote locations, with limited bandwidth and possibly high latency. Early forms of the standard appeared in the late 1990s.
During the normal course of operation, devices or clients using MQTT “publish” their data to a “topic” on a central server, known as a broker. Other devices or clients, “subscribe” to that “topic” on the broker and can pick up the data. The system is quick and allows a one-to-many relationship.
For example, if you publish a “hello, world” message to the “mqtt” topic and there are five clients subscribed to “mqtt,” connected to the same broker, as soon as the data is sent, all five clients will get the message.
Another desirable aspect of MQTT is that a client can “pull” data from the broker. Anytime something is published to a topic, a subscribed client gets the message. The broker can also effectively “push” data to the client. If another client publishes data to a topic on the broker, it’s automatically “pushed” down to any similarly topic-subscribed client connected to that broker.
There’s a great MQTT client-server application, Eclipse Mosquitto, that runs on Linux machines. It’s easy to install and makes distributing data between devices a breeze. Libraries geared for the Arduino (and therefore, the ESP8266) and Python are also available from various sources on the Web. You might want to use Python on a CHIP or a Raspberry Pi to build a monitoring application, grabbing data from the MQTT broker.
You can run MQTT servers and clients on Linux machines, then use libraries to build clients on small devices, like the ESP8266. Data flows seamlessly between all the devices through the common protocol.
Set Up A Mosquitto MQTT Server on the CHIP
To get started with MQTT just set up a quick system on a CHIP computer, using Mosquitto.
I do much of my work on CHIPs using SSH over the network. A fully-operational computer, without a monitor and keyboard/mousepad, that you access remotely over SSH, is typically referred to as “headless.” Although it seems like magic at first, after you get used to working this way it’s no big deal. My WiFi-enabled Raspberry Pi Steampunk Badge is headless and I typically change settings, adjust software and tweak the scripts remotely from a network connected Linux notebook.
Boot up the CHIP computer and make sure it’s connected to your local network. I simply sat the CHIP board on my desk and powered it up with a Samsung cell phone wall-wart attached through a micro-USB cable. I had previously connected to my local area network, so was able to SSH into the device without hassle. The CHIP web page has a little tutorial if you need help.
From a Linux notebook, log into the CHIP using SSH. Insert the appropriate network address for your CHIP.
|
1 |
rob-notebook% ssh -X chip@192.168.1.113 |
I took inspiration for building Mosquitto on a CHIP computer from the “Minimal MQTT: Building A Broker” article on Hackaday. It’s only seven command lines, so you can copy and paste from the web page into your SSH-connected CHIP terminal. Essentially, it’s an apt-get install Mosquitto exercise.
After installation, Mosquitto starts automatically. The install scripts added it to the system so the Mosquitto server also restarted after rebooting the CHIP.
To test the setup enter a Mosquitto subscribe command into a CHIP terminal.
|
1 |
chip-term1% mosquitto_sub -h localhost -t “mqtt” |
The terminal will then patiently sit there until a message is published to the “mqtt” topic. To make that happen, open a second terminal to the CHIP with SSH, then type the following.
|
1 |
chip-term2% mosquitto_pub -h localhost -t “mqtt” -m “hello, world” |
Nearly instantaneously, “hello, world” will pop up in the first terminal window. You can replace the localhost with the CHIP’s IP address and it will still work.
Set Up An MQTT Client On The ESP8266
As I’ve pointed out in past articles, the ESP8266 is easily programmed using the Arduino IDE, so that’s what I use. Lots of Arduino MQTT libraries, of varying complexity exist on the Web. Adafruit has a comprehensive instructional series on setting up and using MQTT servers, clients, hardware and services.
I found a relatively simple client library that works with the ESP8266, by knolleary, on GitHub.
The first step is to download the zip file to your Linux notebook. Pull it into the Arduino IDE by clicking through the Sketch → Include Library → Add ZIP Library menus and selecting your downloaded library file. I renamed the pubsubclient-master.zip, found in my /home/rob/Downloads directory, to pubsubclient.zip, and pulled it into the Arduino library manager.
Once the library is installed in the Arduino IDE, go to File → Examples → PubSubClient to find example programs. Opening mqtt_esp8266, all I had to do was modify the following three lines of code, with my local settings, then upload it to the ESP8266 PIR Yard Sensor for testing.
|
1 2 3 |
const char* ssid = "my-network-name"; const char* password = "my-ap-password"; const char* mqtt_server = "my-CHIP-computer-IP-address"; |
Looking further down in the file, you’ll notice the client_publish and client_subscribe lines, in the reconnect() function. There really isn’t much to using the libraries on the ESP8266. This code simply sends a new “hello world” message to the “outTopic” topic every 2 seconds. I took the liberty of changing “outTopic” to “mqtt” for my tests.
Once the ESP8266 is on the network and sending messages to the CHIP broker, go back to a terminal and subscribe to the topic. The 192.168.1.113 is the IP of the CHIP.
|
1 |
chip-term1% mosquitto_sub -h 192.168.1.113 -t “mqtt” |
A sequential “hello world” message should pop up on the terminal every two seconds.
I also installed Mosquitto on the Linux notebook using the Synaptics application manager. After installation, subscribing to the “mqtt” topic, via a terminal window on the Linux notebook (and pointing to the CHIP MQTT broker) yielded the same message every 2 seconds.
|
1 |
rob-notebook% mosquitto_sub -h 192.168.1.113 -t “mqtt”. |
I sat there and watched both the CHIP and Linux notebook subscriptions update, as the ESP8266 published the “hello world” message to the broker.
The whole thing worked very well. It ran all day without an issue.
What’s Next?
MQTT is a very viable answer to two-way message posting between devices. It’s lightweight and fast while allowing a one-to-many relationship between machines.
Take a look at the Adafruit tutorials, then set up your own system. There are many more features and capabilities, like message delivery after a device was offline, that you might want to check out and use. I’ll certainly use MQTT in future projects, especially with the ESP8266, the Raspberry Pi and the CHIP computer. It should work well with my networked wearables.
Feature image via Pixabay.
