MQTT and the Sparkplug Specification: Enabling Edge-Driven, Best-In-Class Industrial IoT

The Message Queuing Telemetry Transport Protocol (MQTT) has become extremely popular for IoT and Industrial IoT (IIoT) applications, and for good reason. This simple, lightweight, and open transport protocol enables devices and clients to publish information to a central MQTT broker. Other devices, clients, and additional brokers can subscribe to the messages from the MQTT broker and publish their own messages back to the broker.

Before we take a closer look at the benefits MQTT provides, here’s a brief look at the protocol’s origins and relationship to the Sparkplug specification.

An Implementation Standard Is Needed for Interoperability in IIoT

Arlen Nipper and Andy Stanford-Clark (Figure 1) co-invented MQTT while trying to solve a specific problem for Phillips 66 in the late 1990s. The problem they were trying to solve is still relevant today: How do we improve data access in the field for an entire enterprise without using massive amounts of bandwidth?

Figure 1: Arlen Nipper and Andy Stanford-Clark

Instead of following the model that had been used for decades, the team decided to think outside of the typical poll-response industrial automation box. MQTT was the result of that effort.

The majority of implementation information was intentionally left out of the original MQTT specification. This decision enabled maximum flexibility and is the reason Facebook, Amazon Web Services, and Microsoft Azure were early MQTT adopters and have found the protocol so appealing.

However, to meet the interoperability needs of the industrial automation community, MQTT implementation standards had to be defined. This has been achieved with the creation of the Sparkplug specification for MQTT. As Arlen Nipper described in an interview with Inductive Automation in 2019, the Sparkplug specification provides several important benefits for IIoT.

According to Nipper, "Sparkplug is a specification that defines how to use MQTT in a mission-critical, real-time OT environment. It defines a standard MQTT Namespace that's optimized for industrial application use cases. It defines an efficient MQTT Payload definition that's extensible but that's been optimized for SCADA, tag, and metric representations. It defines how best to use MQTT state management in real-time SCADA implementations. And lastly, it defines some high-availability MQTT architectures addressing both redundancy and scale."

But do we really need a specification for MQTT? Won’t a specification reduce the flexibility MQTT offers? I discussed these questions with Walker Reynolds of Intellic Integration. Reynolds is a friend and a superb systems integrator.

The Sparkplug Specification Brings Structure to MQTT

Walker brought some great insights to the discussion. "Yes, MQTT is flexible — you can basically publish any payload to any topic,” he said, “but this can create a mess in the topic namespace. Additionally, building IoT applications using many MQTT clients publishing into the same broker, without any uniformity in the payload structure, takes additional time. Sparkplug B is the specification written by Arlen Nipper and team for publishing industrial data and provides the structure, compression, and uniformity needed to normalize disparate data into a unified structure at the broker."

Walker is right. Sparkplug — B is the latest version of the specification — provides the structure for anyone who wants to create an IoT or IIoT application that uses MQTT without worrying about interoperability between their project and other applications.

Arlen Nipper and his team were wise enough to realize the Sparkplug specification had to be a collaborative effort within the industrial automation community. That’s why they chose to move the specification to the Eclipse Foundation and form the Sparkplug Working Group.

Here are a few of the main reasons to consider an MQTT architecture based on the Sparkplug specification.

Dramatically Simplify Network Architectures

Consider a traditional architecture and the number of connections that must be balanced between devices, servers, and client tools (Figure 2).

Figure 2: Traditional Architecture With Spaghetti-Like Connections

The MQTT architecture is dramatically simpler (Figure 3). You just need to point publishing devices to a single broker and define a single topic, or multiple topics, the device should publish under. Subscribing clients can then use those same topics to filter the data they want to receive from the broker.

Figure 3: Simplified Architecture With MQTT

Use Considerably Less Bandwidth

Because MQTT does not use poll-response communications, it only communicates when change occurs. This massively reduces bandwidth requirements. Additionally, the Sparkplug specification enables MQTT to achieve three times higher compression on the packets it transports than it could previously.

Johnathan Hottell is an application engineer (and genius) with Kelvin Inc, a company that specializes in upstream oil and gas artificial intelligence. Hottell compared bandwidth usage with a Sparkplug MQTT implementation to other industry protocols such as OPC Unified Architecture (UA) and Modbus. Hottell demonstrated that MQTT can provide between 75 percent and 99.5 percent more bandwidth efficiency than Modbus in real-world applications. The savings over other publish/subscribe protocols, such as OPC UA, are also significant (Figure 4).

Figure 4: MQTT Bandwidth Costs Compared to Other Protocols

Source: Johnathan Hottell, Kelvin Inc.

Increase Data Granularity

Because MQTT only communicates changes, you can theoretically increase the scan class without increasing bandwidth usage.

For example, if a discrete tag typically changes six times a day and you’re using poll response, you need to balance how often you scan the tag with the amount of bandwidth you’re prepared to sacrifice. For example, if the tag is polled every 15 minutes, data is communicated 96 times per day to capture six value changes. And there is potential for a delay of up to 14:59 minutes from the time the value changes to the time you are aware of the change. Using MQTT, you are immediately aware of the tag change, and you only send the six value changes per day.

Increase State Awareness

MQTT brokers are aware of client states and can communicate that state to other clients. This awareness is enabled with birth and death certificates and the MQTT Last Will and Testament (LWT) feature.

When a publishing client connects to a broker, it provides a birth certificate and its LWT payload to the broker. The broker then informs MQTT clients that subscribe to the same topic as the publishing client when the publishing client is online or offline. If the publishing client is offline, the broker also provides the subscribing client with the LWT payload. This ensures stale data is not delivered to subscribing clients, and is accomplished using a pre-configured keep alive timer or heart beat check between broker and publishing clients.

Ensure Security

Network security is inherited because MQTT is based on TCP/IP. This means the MQTT specification evolves at the same pace as TCP/IP, and helps ensure the specification remains simple.

Learn More About Sparkplug and Get Involved

If you’re interested in learning more about MQTT and the Sparkplug specification, visit the Sparkplug Working Group website. You’ll find helpful resources and an FAQ page that explains more about the relationship between MQTT, the Sparkplug specification, and Eclipse Foundation projects.

To get involved in developer discussions about Sparkplug, subscribe to the mailing list.