Node Red

Node-RED Example Flow

Introduction

Node-RED is a flow-based, low-code tool for wiring together hardware, APIs and online services. With the IoT platform it serves several purposes:

Acting as a software development kit for crafting and validating the IoT platform's translators and Flow components.
Serving as a simulation tool, facilitating the exchange of data with the IoT platform to explore diverse scenarios.
Providing a verification tool to simulate different integration methods and protocols, streamlining acceptance testing for new releases of the IoT platform software.
Functioning as an integration tool, enabling connections with various third-party services and systems.

However, Node-RED does have its limitations. It demands proficiency in JavaScript programming and operates as a single tenant, thus lacking scalability. Additionally, it offers only basic protection against unauthorized access through a simple login screen, necessitating strict firewall protection.

What you can do (and where to start)

Your goal	Start with
Simulate devices / push test data into the IoT platform	Get started - the MQTT example flow
Read the IoT platform's data over the REST API	Get started - the REST part of the example flow
Build, test and upload translators	Create the IoT platform's translators with Node-RED
Build cross-device data flows (`additionalDeviceUpdates`)	Data Flows

Prerequisites

Node-RED installed on a computer or server (nodered.org).
An account on the IoT platform with access to your server's Swagger UI (https://staging.yggio.net/swagger) - used to get an API token and to reserve MQTT topics / create credentials.
Network access from Node-RED to the IoT platform's MQTT broker and REST API. Keep Node-RED behind a firewall (it has only a simple login).
Basic JavaScript for the REST API and translator work.

Get started

Install Node-RED on a computer or server per the Node-RED instructions. The easiest way to exchange data with the IoT platform is the standard MQTT in / MQTT out nodes pointed at the IoT platform's MQTT broker - from there you can build flows and dashboards and simulate data sets. Using the IoT platform's REST API instead is more powerful but needs more JavaScript.

Fastest first success. The example flow below is complete (6 simulated devices, bidirectional MQTT, plus REST) - great for learning, but a lot at once. If you just want to see one data point land in the IoT platform first: reserve one MQTT topic (the IoT platform, step 5 below), add a single MQTT out node pointing at it, and publish a small JSON message like {"temperature": 21}. A device appears in the IoT platform within a minute. Then import the full flow for the complete picture.

For a swift start, import the following example Node-RED flow into your Node-RED installation:

Node-RED Example Flow JSON

The example flow demonstrates how to use MQTT to publish simulated data and subscribe to it from the IoT platform's MQTT broker. Additionally, it includes an example of logging in with the IoT platform's REST API, saving a token, and performing a GET request to /iotnodes and a time series.

Do the following configurations in the IoT platform:

Go to the IoT platform's Swagger.
Retrieve a valid token from the IoT platform by navigating to the /auth/local endpoint. Click 'Try it out,' enter your username and password for the IoT platform, click 'Execute,' and copy the response token to your clipboard.
Click the green 'Authorize' button in the top right, paste the token into the textbox, and click 'Authorize.' You are now logged into the IoT platform's API in Swagger and can try out all the APIs.
Go to the /basicCredentialSet endpoint and create a basic credential set with a suitable strong username and password.
Go to the /reservedMqttTopic endpoint, use the basicCredentialSet, and reserve 6 different topics for 6 IoT nodes in the IoT platform. The topic must follow the structure yggio/generic/v2/[youruniquedeviceid]

Do the configurations in Node-RED:

Import the example flow.
In the top left 'Start' node, update the username and password to match your credentials on the IoT platform to enable the use of the REST API.
In one of the MQTT out nodes that represent devices on the IoT platform, add a new MQTT server with the same URL as the IoT platform's URL, and enter your username and password in the 'Security' section.
Add the topic for device 1 that was created in step 5 above.
Repeat the above steps for the remaining 5 MQTT out nodes, using the MQTT server created earlier.

You can now deploy the flow and test the REST API integration GET /iotnodes by clicking the start button in the top left. You will get an error message from the time series data but disregard it for now. If you go back to your account on the IoT platform, there should now be one MQTT node receiving simulated data at regular intervals.

Do the final configuration to get bi-directional communication:

Go to your account on the IoT platform and click on the new MQTT node.
Go to the 'Channels' tab and create a new MQTT channel. Choose the basicCredentialSet type of channel and refer to the basicCredentialSetId created earlier.
Copy the complete topic from the MQTT channel to the clipboard.
Return to Node-RED, locate the node named Device1-FromYggio, and click on it. This is an MQTT in node.
Select your existing MQTT server and paste the topic copied from the IoT platform into the topic field.
To get the time series API call to work, update the IoT node id in the lower of the two 'Prepare API call' to the _id in the 'General' tab in the Device list of your new MQTT node.
The configuration is now complete, and you can deploy it. Within 1 minute, another 5 MQTT IoT nodes will be created in the IoT platform. You can observe the data being generated in Node-RED, sent to the IoT platform by publishing it on the IoT platform's MQTT broker, subscribed to from the IoT platform's MQTT broker, delayed, and then re-published on the IoT platform's MQTT broker. You will also see the time series API now starts to work.

To see the simulation in action, go to the Device list in the IoT platform. Use 'Select many' to select the new MQTT nodes, go to Charts, and examine the different fields from the simulation. With Node-RED and some technical JavaScript programming skills, you can quickly integrate the IoT platform with various systems, create and verify the IoT platform's translators, and develop the IoT platform's Flow components. You can also import this nice Device list view to see it work in real time: Node-RED Device view.txt

Create the IoT platform's translators with Node-RED

The basics

Node-RED makes a great SDK for building and testing the IoT platform's translators. Both run on Node.js - the same JavaScript engine - so a translator that works in a Node-RED simulation (and uses strict, valid JavaScript) will behave the same once uploaded to the IoT platform via the translator API in Swagger.

Two caveats when moving code from Node-RED to the IoT platform.

Sandbox surface. A Node-RED function node has full Node.js; the IoT platform's translator sandbox is narrower - it provides standard JavaScript (Date, Math, JSON), lodash (_), Buffer and console.log, but no require, timers, network or filesystem. Keep your translate to those.

Data-model consistency. A translator's output must use the canonical field names of the IoT platform, units and quantities - that is what makes dashboards, alarms and downstream translators work across every device. Before writing output fields, read Translator Development and the field table in the Translator API reference.

Start developing

Node-RED my first translator

Node-RED my first translator JSON

Import the flow above into Node-RED. It consists of:

An inject node.
A function node that generates simulated data.
A function node that implements a translator and returns the result.
A debug node to make the data visible.
A flow to stringify the translator and the specification, which gets printed in the Node-RED console log.
A specification describing the translator and its output. The specification's field names and data types must exactly match the translator's output; otherwise validation fails and no result is stored.

This is a working translator you can upload via the API and attach to any device that provides rssi and snr. It computes a signal-strength value: rssi + snr when snr is negative, otherwise rssi alone. The spec is a contract with the end user and must match the translator's output exactly - any deviation causes validation to fail and the result is discarded. For dynamic or variable results, return them inside a JSON object field.

For complete, copy-pasteable templates (hardware, calculate, set-alarm, analytics) and a local test harness, see Translator Development. This page focuses on the Node-RED-specific workflow of building and simulating them.

Hardware decoders

If the manufacturer provides a reference JavaScript decoder, building and verifying the translator is usually quick. Reuse the same translator flow: paste the manufacturer's decoder into the flow and call it from translate, passing it the hex payload.

Keep the manufacturer's decoder verbatim, then harmonize as the final step: map its raw output to the IoT platform's canonical field names, scaling units accurately. Do not return the raw decoder output directly - raw vendor names like TempC_SHT or BatV are not canonical fields. In the example below rawTranslate is the manufacturer's decoder:

function translate ({encodedData}) {
  const {hexEncoded, port} = encodedData;

  if (!hexEncoded || !port) {
    throw new Error('Expected fields hexEncoded and/or port are missing');
  }

  const decoded = rawTranslate({hexEncoded, port}); // manufacturer decoder, kept verbatim

  // Harmonize: rename raw vendor fields to canonical Yggio fields and scale
  // units to the canonical unit (e.g. mV -> V, hPa -> Pa).
  return {
    result: {
      temperature: decoded.TempC_SHT,        // vendor name -> canonical field
      relativeHumidity: decoded.Hum_SHT,
      batteryVoltage: decoded.BatV / 1000,   // mV -> V
    },
  };
}

To verify the translator, you need real payloads and the expected results for each payload. Manufacturers usually provide examples that can be used to create simulated data for the translator. Alternatively, you can deploy a real device in the IoT platform and use the payloads received by the IoT platform as simulated data for the translator.

If no reference decoder is available from the manufacturer, writing a decoder becomes a much larger task, potentially taking anywhere from a few hours to weeks of development. This process often involves complex handling of bits and bytes, and a variety of different payloads will be needed to achieve a satisfactory level of verification.

Data Flows

The IoT platform's translator model enables a translator to send translation results to other IoT nodes via 'additionalDeviceUpdates', provided that all administrators of a device have write access to the targets. This feature allows for the management of very complex use cases and real-time visualization of enriched data, as target nodes can be dynamically determined depending on the translation. The IoT platform also supports atomic updates of aggregation nodes, ensuring that regardless of the order in which data arrives, the result will always be accurate. Since Node-RED supports the creation of flows, however not dynamically as the IoT platform, it is an excellent tool for developing and verifying translators that implement complex and dynamic data flows.

Principle to set up data flow simulations

Develop the translators with 'additionalDeviceUpdates' using the standard method described above, by simulating expected input data and forwarding it to the translators.
Once the translator is ready, upload it to the IoT platform via the translator API.
Copy the original flow example described at the top of this page and add the newly developed translator to the MQTT nodes.
Create any necessary 'Generic nodes' that can be used to share data to.
Inject data with Node-RED into the simulation via the IoT platform's MQTT broker. If necessary, create multiple nodes with the translator and verify all expected behaviors.
Subscribe to data from MQTT nodes and use it to develop any additional required translators to manage the complete data flow.
A key item is "additionalDeviceUpdates" need the identifier of the target node, an easy way to make the identifier configureable is to use the contextMap.

Example of additionalDeviceUpdates:

function translate(iotnode) {
  const log = _.get(iotnode, 'log');
  const measurementFields = _.get(iotnode, 'contextMap.measurementFields','').split(',');
  const additionalDevice = _.get(iotnode,'contextMap.targetNodeSecret');
  const sourceDevice = _.get(iotnode,'name','unknown').replace(/\W/g, '-');
  let measurements = {};
  let result = {};

    // Transfer selected measurement fields to target node
  if (additionalDevice != undefined && measurementFields != undefined) {
    for (let i = 0; i < measurementFields.length; i++) {
      measurements[measurementFields[i]] = { [sourceDevice]: _.get(iotnode, measurementFields[i]) };
    }
    if (log != undefined && log.message != '')
      result = {
        ...result,
        'log': {
          'message': '',
          'type': '',
          'priority': '',
          'category': ''
        }
      };
  }
  else if (log == undefined || log.message == '')
    result = {
      'log': {
        'message': 'At least one of the 2 fields targetNodeSecret or measurementFields are missing in the contextMap.',
        'type': 'Error',
        'priority': 'Low',
        'category': 'Status'
      }
    }
    return {
      result,
      additionalDeviceUpdates:
        [{
          identifier: {
            secret: additionalDevice
          },
          result: { measurements }
        }],
    };
  }

Debugging

Alright, so your translator performs flawlessly in the Node-RED simulation, but once uploaded to the IoT platform, nothing seems to happen when the node it was added to updates. Without any clear actionable information, you might be wondering what to do next. There are three potential types of issues that could be causing this:

First, check the device's Logs. When a translator crashes, the IoT platform records the error in the device's Logs and keeps it for 6 hours. Open the Logs and filter Type = Debug and Category = System to see it, then copy the failing input into your Node-RED simulation to reproduce it. A successful translation writes no log, so an empty log means it either ran fine or never triggered - check the device's Last reported column in the device list to tell which (if it updated, the translator ran).

Validation of Translator Output vs. Specification Fails, this is the most common reason and it's easy to overlook. Make sure to:

Review all possible output fields and their data types, ensuring a 100% match vs the specification, including case for every single character.
Enforce data types using functions like Number(), String(), and Array.isArray(myArray) to prevent discrepancies in data types.
Confirm every output field is a canonical field name of the IoT platform (or a properly-specified device-specific one) - a mistyped or non-canonical name is a silent mismatch downstream.

Translator Crashes During Execution: If the translator crashes, it's likely due to inaccurately handling of data input data combinations. The most common reason is trying to assign or reference an undefined variable.

Copy the input data sent to the translator from the IoT platform and add it to your Node-RED simulation. This will reveal if the translator crashes or not.
Check you are not using anything outside the IoT platform's sandbox (require, timers, Node APIs) that happens to work in Node-RED but not in the IoT platform.

Translator Fails to Compile in the IoT platform: If the translator fails to compile in the IoT platform, the issue may be related to a missing semicolons or other code issues after stringification.

To address this, run the code through linters to check if it's valid JavaScript and JSON. There are plenty of online Javascript validators available by searching on "Javascript online linter validation".
If your translate calls helper functions, make sure they are included in the uploaded code - the whole code string runs in the sandbox, so a helper left behind throws "X is not defined".

Introduction​

What you can do (and where to start)​

Prerequisites​

Get started​

Create the IoT platform's translators with Node-RED​

The basics​

Hardware decoders​

Data Flows​

Debugging​

See also​