MQTT stands for Message Queuing Telemetry Transport. It is a lightweight protocol designed for efficiently transporting messages between M2M or IoT devices.
It uses a publish-subscribe model where devices publish information to a broker, and any number of clients can subscribe to them.
For example, a temperature sensor device in the home could publish the temperature to a topic, for example /home/indoor/temperature. The air conditioning system could then subscribe to this temperature and switch on and off when the value exceeds programmed thresholds.
It might also be desirable to have an indoor temperature display. This device could also subscribe to the /home/indoor/temperature topic and display the temperature.
Brokers can also be cascaded or bridged. An edge broker can subscribe to a topic or topics published by another broker upstream and republish them. For example, the national electricity market could publish the electricity spot price. The local home-based broker publishing the indoor temperature in the example above, could subscribe to the electricity spot price and the air conditioner could via the local broker throttle back based on the market price.
Brokers generally only distribute information, they do not store data for later analysis. A middleware ‘connector’ can be used between the broker and a database to subscribe to topics and copy this data into the database. Hence, if there was a use case to also store the temperature, some middleware software could subscribe to the /home/indoor/temperature and store the data for later analysis.
MQTT was first used to monitor oil pipelines within a SCADA network. The battery operated devices needed to be lightweight, and as the data was being sent over satellite, the protocol needed to be efficient. Today, MQTT is powering products from home automation to software applications like Facebook messenger.
The MQTT Standards can be found at Oasis:
Topics
At the heart of MQTT is topics. These are dynamically generated by the clients and hence, the broker doesn’t need to be configured with a complete list.
- Topics are divided into multiple levels/hierarchy separated by a forward slash.
- Topic names and filters are case sensitive.
- Topic names and filters can include the space character, but must not include a null character.
- All topic names and filters must be at least one character long.
- A leading or trailing forward slash creates a distinct topic name or filter. e.g. home/temp is a different topic to home/temp/
- Topic names and filters are encoded in UTF-8 and must not exceed 65535 bytes long.
Filters
Wildcards can be used to subscribe to multiple topics.
The hash character ‘#’ is used to match multiple levels. It must be specified either on its own (subscribe to all topics) or following a topic separator. For example home/indoor/# is valid, however home/indoor# is not.
The plus sign ‘+’ only matches a single level. For example home/+/temperature.
QoS
MQTT supports three Quality of Service (QoS) levels. This specifies if a message has guaranteed delivery.
- QoS 1 ‘At most once delivery’ – Best effort delivery, no response is sent from the receiver and no retry is performed by the sender. This is the most efficient.
- QoS 2 ‘At least once delivery’ – Ensures the message is received at least once, however multiple duplicate messages are possible.
- QoS 3 ‘Exactly once delivery’ – Highest QoS and incurs the most overhead.
Protocol
The native MQTT protocol uses TCP as the transport layer. TCP port 1883 is used for clear text transmission, and TCP 8883 for MQTT over TLS.
The payload is application specific and supports binary transfer. The standard allows for a maximum payload size of 256MB, although some brokers and/or clients may restrict this.
More often than not, the payload is text. It is up to the developer on how to structure the payload. Some devices will report a single variable per topic, while others may use a JSON format or similar and include multiple parameters.
Installing the Mosquitto Broker
Eclipse Mosquitto is one of the more popular open source MQTT brokers and is super easy to set up.
To install the Mosquitto MQTT broker and clients on a Linux platform such as ubuntu, run:
# sudo apt install mosquitto # sudo apt install mosquitto-clients
The clients package is useful for debugging purposes.
At this stage, you should have a working MQTT broker!
You can test it out by subscribing to a topic (-t) such as dev/test:
# mosquitto_sub -t dev/test
Then in another terminal, publish a message (-m) to the dev/test topic:
# mosquitto_pub -t dev/test -m "Hello World!"
This should result with a Hello World message appearing in the mosquitto_sub process.
However, any attempts to access the broker except from the localhost (127.0.0.1) interface will result with the message Error: Connection refused. A look in the log file should indicate the following
Starting in local only mode. Connections will only be possible from clients running on this machine.
Create a configuration file which defines a listener to allow remote access.
For more details see https://mosquitto.org/documentation/authentication-methods/
Mosquitto Security
The out of box experience doesn’t include any security or encryption. However, depending upon your use case this might be perfectly adequate, for example, running a home automation system on your private firewalled LAN.
To make Mosquitto a little more secure, it can be good practice to enable authentication. It should, however, be noted the username and password is sent in clear text unless you also use TLS encyption.
First, create (-c) a password file using mosquitto_passwd with a user called <username>. It should prompt you to enter a password.
# sudo mosquitto_passwd -c /etc/mosquitto/passwd <username>
Then edit mosquito’s configuration file:
# sudo nano /etc/mosquitto/mosquitto.conf
Add the following three lines towards the bottom of the file:
password_file /etc/mosquitto/passwd allow_anonymous false listener 1883
Now restart the service to reload the configuration file:
# sudo systemctl restart mosquitto
If you now subscribe or publish messages using mosquitto_sub or mosquitto_pub as shown above, it should return a “Connection Refused: not authorised.”
You will now need to pass a username (-u) and password (-P) e.g.
mosquitto_sub -d -u <username> -P <password> -t dev/test
mosquitto_pub -d -u <username> -P <password? -t dev/test -m "Hello World"
Enabling TLS security will ensure usernames, passwords and the payload is encrypted. However, the tradeoff is increased complexity at both the broker and client side with managing certificates. As many sensors are small microcontroller based systems with non-security sensitive payloads communicating over local networks, this increased complexity is often unwarranted and a hindrance. However, if you were communicating over the public internet, it would be strongly recommended.
Example Code
With your MQTT broker functioning and moderately secure, the next step is to start writing code to subscribe and/or publish messages.
Eclipse Paho is perfect for this with support for Java, Python, JavaScript, GoLang, C, C++, Rust, .Net (C#), Android Service and Embedded C/C++.
Visualisation
For users wanting to save data into a database for graphing, a common time series database is InfluxDB. It can use a connector called Telegraf to obtain data from MQTT. Once data is contained within Influx, Grafana can be used to visualise it.
Be the first to comment