Skip to main content

Protocols

A protocol is a set of rules for how data moves between machines. Without shared rules, two devices cannot understand each other, much as two people speaking different languages cannot without an interpreter.

Protocols define how data is split into packets, how errors are detected and corrected, how connections are opened and closed, and what shape messages take.

Anatomy of a packet

Almost every packet, at every layer, has the same two parts:

A packet in two parts: the header, carrying addresses and control information, meaning where the packet comes from, where it is going and how the receiver should handle it; and the payload, carrying the data being carried, meaning the sensor reading, the command or the response, the part the application cares about

Concretely, the header holds the source and destination as an IP address and port, plus sequence numbers, error checking and the message type.

Layers nest. An MQTT message is the payload of a TCP segment, which is the payload of an IP packet, which is the payload of whatever the radio sends. Each layer adds its own header and reads only its own, which is what lets the same MQTT message travel over WiFi in one building and cellular in the next.

How the protocols stack up

The horizontal IoT platform with LwM2M for device management and telemetry, above the application protocols HTTP for REST APIs, MQTT for publish and subscribe and CoAP for constrained devices, above the transports TCP carrying HTTP, MQTT, raw TCP and LwM2M over TCP and UDP carrying CoAP, raw UDP and LwM2M over UDP, above IP for addressing and routing, above the link and physical medium of Ethernet, WiFi, LTE, NB-IoT, LTE-M, 5G, LoRaWAN, Bluetooth, BLE, Z-Wave and ZigBee

Lightweight protocols such as CoAP and LwM2M make efficient communication possible for small battery-powered devices, while MQTT and HTTP give reliable transport over TCP.

IP: addressing and routing

The Internet Protocol is what makes the internet one network rather than many. It gives every endpoint an address and routes packets between them, without caring what medium carries them.

  • IPv4 addresses look like 192.168.1.10. There are about four billion, which ran out, so most devices sit behind network address translation.
  • IPv6 addresses look like 2001:db8::1. There are enough for every device many times over, and cellular IoT networks increasingly use it.

IP itself makes no promises. It does not guarantee that a packet arrives, that packets arrive in order, or that they arrive only once. Everything above it either adds those guarantees or decides it can live without them. That decision is the difference between TCP and UDP.

TCP: reliable and ordered

TCP establishes a connection before any data flows, using a three-way handshake, and tears it down afterwards. Within the connection it numbers every byte, acknowledges what arrived, retransmits what did not, and delivers the result to the application in the order it was sent.

Use it when correctness matters more than overhead, which is most of the time. HTTP, MQTT and LwM2M over TCP all rely on it.

What it costs:

  • Setup takes a round trip before any data moves, and the encryption handshake takes more.
  • Keeping the connection open costs power on a battery device, because the radio must periodically wake.
  • Head-of-line blocking: one lost segment delays everything queued behind it.

UDP: minimal and fast

UDP sends a datagram to an address and port and stops caring. There is no connection, no acknowledgement, no retransmission and no ordering. A datagram arrives intact or not at all.

That sounds worse and often is not. For a sensor sending a temperature every ten minutes, a lost packet is not worth a retransmission because a fresher value is coming anyway. Removing the connection removes the handshake, the keep-alive and the state on both ends, all of which are battery and memory a constrained device does not have.

Use it for small, frequent, individually expendable messages. CoAP, raw UDP telemetry and LwM2M over CoAP all use it.

UDP: an IoT sensor or system sends messages straight to the server, with no handshake and no delivery acknowledgement

HTTP: request and response

HTTP is the foundation of the web, of REST APIs and of gRPC, which uses HTTP/2 as its transport. In IoT it is how most systems talk to each other, and how many devices report in.

A client sends a request, and the server returns a response. The request names a method and a path; the response carries a status code and usually a body.

An HTTP exchange: the client sends a request, GET /iotnode/stats, to the web server, which returns a response of 200 OK plus the response data

GET /iotnode/stats/{id} HTTP/1.1
Host: yggio.example.net
Accept: application/json
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 85

{
"sensorId": "temp-001",
"timestamp": "2025-10-08T10:30:00Z",
"temperature": 22.5,
"unit": "C"
}

Methods

REST APIs use the HTTP methods to mean specific things:

MethodMeansTypical use
GETRead, changes nothingFetch a device, list devices, read history
POSTCreate, or submitCreate a device, post a reading
PUTReplace the whole thingOverwrite a device document
PATCHChange part of itUpdate one field
DELETERemove itDelete a device

GET is safe to repeat and safe to cache. PUT and DELETE are idempotent, meaning doing them twice leaves the same result as doing them once. POST is neither, which is why a retried POST can create two of something.

Status codes

The first digit tells you who to talk to about it:

RangeMeaningCommon examples
2xxSuccess200 OK, 201 Created, 204 No Content
3xxRedirection301 Moved Permanently, 304 Not Modified
4xxThe request was wrong400 Bad Request, 401 Unauthorized, 403 Forbidden, 404 Not Found, 429 Too Many Requests
5xxThe server failed500 Internal Server Error, 502 Bad Gateway, 503 Service Unavailable

A 4xx means fix the request: the URL, the credentials, the permissions or the body. A 5xx means the request was reasonable and the other end could not fulfil it, so retry with a delay.

Headers and security

Headers carry everything that is not the body: Content-Type says what the body is, Authorization carries the credential, Accept says what the client can read. HTTPS is HTTP inside TLS, which encrypts the whole exchange including the headers, and is the only form that should cross a public network.

What HTTP is less suited to

HTTP is a client asking a server. If the server has something to say first, it must wait to be asked, so a device that needs to receive commands has to poll, which costs battery and adds latency. That is exactly the gap MQTT fills.

MQTT: publish and subscribe

MQTT is a lightweight protocol built for exactly the situation IoT is in: many small devices, on unreliable networks, behind firewalls, sending small messages. Together with HTTP it is the most important protocol in modern IoT.

Instead of asking a server, a device publishes a message to a topic on a broker. The broker forwards it to everyone who subscribed to that topic. Publishers and subscribers never know about each other, which is what makes the pattern scale.

MQTT publish and subscribe: an IoT sensor or system publishes to a topic on the MQTT broker, and the broker forwards the message to every subscriber of that topic

Why it fits IoT so well

Two properties quietly solve problems that cost real money.

The session is always opened by the device outwards, so there are no inbound ports to open and a whole category of firewall argument disappears. And the overhead is low enough that thousands of devices share one broker comfortably.

An MQTT integration usually needs both ends to agree only on the data model rather than on an entire API, which is why it has become the common language between systems, not just between devices.

Topics

A topic is a path, and it is the whole addressing scheme:

Yggio/output/v2/<credentials id>/<node id>

Subscribers can use wildcards:

  • + matches exactly one level. building/+/temperature matches building/floor1/temperature and building/floor2/temperature.
  • # matches everything below, and must come last. building/# matches every topic under building.

Design topics as a hierarchy from general to specific, so that a subscriber can choose how much to take.

Quality of service

MQTT offers three delivery guarantees, chosen per message:

QoSGuaranteeCostUse for
0At most once, fire and forgetLowestFrequent readings where the next one is due shortly
1At least once, may duplicateOne acknowledgementMost telemetry, where losing a value matters
2Exactly onceFour-part handshakeCommands and billing events, where a duplicate would be harmful

QoS 1 is the usual choice. It can deliver the same message twice, so anything acting on it should tolerate a repeat.

The other features worth knowing

  • Retained messages: the broker keeps the last message on a topic and gives it to each new subscriber immediately, so a dashboard shows a value on connect instead of waiting for the next report.
  • Last will and testament: the device registers a message when it connects, which the broker publishes if the device disappears without saying goodbye. This is how you detect a device that dropped rather than one that logged off.
  • Keep-alive: a heartbeat interval agreed at connect. If the broker hears nothing within it, the connection is considered dead and the will is published.
  • Clean and persistent sessions: a persistent session lets the broker hold subscriptions and queued messages while the device is offline, so nothing is missed across a gap.

Security

MQTTS is MQTT inside TLS, normally on port 8883, and is what should be used anywhere outside a trusted network. Authentication is by username and password or by client certificate, and authorization is per topic, so a device can be permitted to publish only its own topic and nothing else.

Yggio ships a fully compatible MQTT broker with access control built in, and it does something more than forward messages. Data published to it is processed in Yggio's data flows and stored as time series at the same time, so every message is immediately available for history, analysis and visualization without anyone building a second path for it.

Yggio as an MQTT broker: published data goes to the broker, into data flows and into time series storage at once, reaching both subscribers and analysis and visualization, with access control retained throughout

CoAP: REST for constrained devices

CoAP, the Constrained Application Protocol, was designed for machine to machine communication on devices with very little memory, processing power and battery.

It keeps the model developers already know from HTTP, with resources at paths and the methods GET, POST, PUT and DELETE, and rebuilds it on UDP with a compact binary header measured in single-digit bytes rather than the hundreds a set of HTTP headers can take.

A CoAP exchange: the IoT sensor acting as client sends a request, GET /iotnode/stats, to the server, which returns a status code plus response data, using a compact binary format over UDP

What it adds on top of UDP:

  • Confirmable and non-confirmable messages. A confirmable message is acknowledged and retransmitted if it is not, which gives reliability where it is wanted without a permanent connection.
  • Observe, which lets a client register interest in a resource so the server sends updates as the value changes. This is the push that plain HTTP lacks.
  • Block-wise transfer, which moves payloads larger than a datagram in pieces, used for firmware images.
  • Discovery, so a client can ask a device what resources it offers.

Security is DTLS, which is TLS adapted for datagrams. OSCORE is an alternative that protects the message itself rather than the channel, so it survives being relayed by a gateway that would otherwise have to terminate the encryption.

CoAP is common on NB-IoT and LTE-M devices, and underneath LwM2M.

LwM2M: managing the device, not just its data

Everything above moves measurements. LwM2M, from the Open Mobile Alliance, manages the device itself, and normally runs over CoAP, sometimes directly over UDP or TCP.

It defines a standard structure of objects and resources, each with a registered number, so that "battery level" or "firmware version" means the same thing on every device that implements it. That standardization is the point: device management stops being manufacturer-specific.

The lifecycle it covers:

  • Bootstrap, where a device is given the address and credentials of its management server.
  • Registration, where it announces itself and the objects it supports.
  • Device management and configuration, reading and writing those resources remotely.
  • Firmware update over the air, using block-wise transfer.
  • Telemetry, reporting values on a schedule or on change.

Choosing between them

ProtocolTransportPatternBest suited to
HTTPTCPRequest and responseSystem to system, REST APIs, occasional reporting from mains-powered devices
MQTTTCPPublish and subscribeContinuous telemetry, commands to devices, system to system streaming
CoAPUDPRequest and response, plus observeConstrained battery devices, cellular IoT
LwM2MCoAP, UDP or TCPDevice management and telemetryFleets needing standardized remote configuration and firmware update
Raw TCP or UDPTCP or UDPWhatever the device doesLegacy and bespoke equipment

In practice a single estate uses several at once, which is normal and is precisely what the platform layer is there to absorb.

Security across the stack

Encryption is applied at every hop rather than once. A LoRaWAN reading, for example, is encrypted with AES-128 within LoRaWAN itself from the device to the network server, then travels inside TLS from the network server to the platform, then inside TLS again over MQTTS or HTTPS out to the user.

The principle to carry away: each layer protects its own hop, and a gateway that terminates one protocol and starts another is a point where the data exists in the clear unless something like OSCORE protects the message end to end.