Sending Sensor Data From Arduino to MQTT

ByZach
thumbnail for the blog post on sending Arduino sensor data to mqtt

Sending sensor data from an Arduino to an MQTT server is a very efficient way to share data from one Arduino to another Arduino, Node-Red, or any other device that can read MQTT data. In this article we’ll jump into sending data, if you want to retrieve data from MQTT, check out this post.

JUMP TO FINAL CODE

SOURCE FILES

What is MQTT?

MQTT is a communication protocol that allows a client to publish data to a broker (server). From there, one or more clients can subscribe to the data. A client can publish, subscribe, or both, so data can easily flow multiple directions. MQTT differs from HTTP request in that data isn’t directly send from one device to another, but rather only has to be sent once to a central location and then can bread by as many devices as desired. Both do more or less the same thing, but MQTT is a better alternative in larger scale applications.

Brokers

There are multiple different brokers out there, but the most common one is Mosquitto. Mosquitto can be run on multiple platforms, but the most common is to run it on a raspberry pi. Mosquitto pairs nicely with Node-Red to connect other IoT (Internet of Things) devices. I have them both installed on the same raspberry pi to for easy communication.

Topics

MQTT data is published to what are called topics. Topics are basically just file paths that the data is saved to. I like to think of this as a location broken down by rooms. Each room is separated by a /. For instance, a topic for a specific room in a house could be home/room1/sensorData. Then the next room would be home/room2/sensorData. Each topic hold 1 set of data, so it important to keep them unique! As you will see below, I set my base topic, and then append to the end of it for each sensor’s data or other parameter being published.

Quality of Service

MQTT has 3 different quality of service levels from 0-2. The PubSubClient we will be using in our Arduino code is only capable of publishing level 0 data and subscribing up to level 1 quality data. No worries though and this doesn’t effect the ability to read level 2 data, it just doesn’t guarantee level 2 quality.

  • Level 0 – Will send the data once, but does not guarantee the data is received by the intended recipient.
  • Level 1 – Will send the data until it is received by the intended recipient, but can send more than 1 time
  • Level 2 – Will send data only once and confirm that is it is received by the intended recipient.

As you can see, even level 0 quality is often adequate.

Message Retention

When publishing a message to MQTT it can be specified if the message is retained by the broker. I like to have messages retained, especially if using a QoS of 0 so that when a new client subscribes to a topic it will automatically get the latest data that was sent by the publisher to that topic. If this is not important, the message retention can be set to false so the data is only sent once, and only clients already subscribed to the topic will get it.

Setting up the Arduino to Send Sensor Data to MQTT

Hardware Requirements

For this example there are only 2 hardware requirements to get this up and running. I am going to be showing you how to get this setup on an Arduino Uno, but this would also work on an Arduino Mega or really any other Arduino device. The only consideration is making sure the device has enough memory to run the sketch.

Software Requirements

All of this code is going to be written in the Arduino IDE. There are a couple of libraries that you will need as well.

Wiring the Arduino and Sensors

In this section I am going to go over how I hook up the sensors I am using, but this will vary depending on your setup. Feel free to skip to code.

To wire up the plant sensors to the Arduino I did have to use logic level converters. These converters take the 5v logic from the Arduino, and convert it to the 3.3v logic that the sensors need. It is super easy to wire these in, just connect the HV side to 5v and ground from the Arduino, and the LV side to 3.3v from the Arduino. Only 1 side needs to have the ground connected. Beyond the voltage part, it is as simple as wiring from a pin to an input, and from and output to the sensor.

Fritzing diagram of an Arduino uno, ethernet shield, and logic converters used to hookup plant monitoring sensors.

Choosing the right Pins on the arduino

Since the sensors I am using run over serial, it is important to think about which pins to use, as not all the pins support hardware or software serial. On the Uno, there is only 1 set of hardware serial pins, but on the Mega there are 4. Because of this, it may be better to go with Software Serial on the Uno to drive the 4 sensors, and a combination of both Software and Hardware Serial on the Mega as I’ll explain below.

Arduino Uno

On the Uno there is only 1 set of Hardware Serial pins (Pins 0 and 1) and these are shared by the USB bus which is used to upload a sketch from the programmer (Arduino IDE). Because of that, I chose to use 4 sets of pins for Software Serial.

To use Software Serial, the digital pins have to support interrupts for the RX part of the serial communication. To see which pins are capable of this, look at page 3 of this pinout from Arduino and make sure the pin has a PCINT label next to it. All of the analog and digital pins are capable of this on the Uno, so it really doesn’t matter which one are used. The only consideration is that the ethernet shield on the Uno uses pins 4, 10, 11, 12, and 13. For this reason, I chose pins 6, 7, 8, 9, 14, 15, 16, and 17. The Analog pins (pins 14-17) are used for the RX part of the serial communication, and pins 6-9 are used for the TX part of the serial communication.

Arduino Mega

On the Mega there are 4 sets of Hardware Serial pins. Serial0 (Pins 0 and 1) is shared with the USB bus which is used to upload a sketch from the programmer (Arduino IDE). I have 4 sensors I want to have in an array, so I opted to go with all Software Serial instances to make things streamlined.

To use Software Serial, the digital pins have to support interrupts for the RX part of the serial communication. To see which pins are capable of this, look at pages 2 and 3 of this pinout from Arduino and make sure the pin has a PCINT label next to it. Another consideration is that the ethernet shield on the Mega uses pins 4, 10, 50, 51, 52, and 53. For this reason, I chose pins 46, 47, 48, 49, 62, 63, 64, and 65. The Analog pins (pins 62-65) are used for the RX part of the serial communication and pins 46-49 are used for the TX part of the serial communication.

Code to Send Sensor Data From Arduino to MQTT

Initial Setup

Starting out with a blank sketch, the first thing to do is import the required libraries. They are highlighted in the code block below.

#include <SPI.h>
#include <Ethernet.h>
#include <PubSubClient.h>
#include <SoftwareSerial.h>

void setup() {
  // put your setup code here, to run once:

}

void loop() {
  // put your main code here, to run repeatedly:

}

Sensor Setup

The first thing to do is setup the sensors being used. As I said in the prior section, I am going to be using Software Serial for this. The first two parameters that need to be define are the total number of sensors being used (totalSensors) and the starting number of the first sensor (startSensorNum). This is going to be more important when working in an environment where multiple Arduinos are sending sensor info to MQTT. EX. Arduino 1 has sensors 1-3 connected to it and Arduino 2 has sensors 4-6 connected to it. The starting sensor number for Arduino 1 would be 1, and for Arduino 2 it would be 4.

Next define the 4 sensors and the pins they’re using. After that, an array can be defined that includes all of the sensors.

After all of the parameters are define everything needs to be initialized in the setup block. The first item to initialize is Serial for our debug printout. 9600 is the standard baud rate, but other rates can be used if desired. Do the same thing for the 4 sensors, also with a 9600 baud rate.

#include <SPI.h>
#include <Ethernet.h>
#include <PubSubClient.h>
#include <SoftwareSerial.h>

//DEFINE SENSORS
const int totalSensors = 4;
const int startSensorNum = 1;

SoftwareSerial sensor1(14, 6); //RX, TX
SoftwareSerial sensor2(15, 7); //RX, TX
SoftwareSerial sensor3(16, 8); //RX, TX
SoftwareSerial sensor4(17, 9); //RX, TX

SoftwareSerial* sensors[] = {&sensor1, &sensor2, &sensor3, &sensor4};

void setup() {
  
  Serial.begin(9600);

  //Initialize Sensors
  sensor1.begin(9600);
  sensor2.begin(9600);
  sensor3.begin(9600);
  sensor4.begin(9600);

}

void loop() {
  // put your main code here, to run repeatedly:

}

Ethernet Setup

Moving on, the next thing to setup is our network information. There are 3 pieces of data needed for this – the MAC address, the Arduino IP address, and the MQTT Server address.

  • MAC address – This should be printed on the back of the ethernet shield. If not, it can be made up or randomly generated.
  • Arduino IP Address – This is optional, but I highly recommend setting a static IP for your Arduino. If you choose not to use a static IP, uncomment the line labeled in the code below.
  • MQTT Server IP Address – This is the IP address of your MQTT server.

Under where the sensors were defined, define the Mac address being used, and the two IP addresses from above. If you’re planning to use DHCP instead of a static IP for the Arduino, feel free to comment out the line for ip. After this, define the EthernetClient.

In the setup loop initialize the network below where the sensors were initialized. Add a small delay to allow time for the network to connect before moving on to the next section of code.

#include <SPI.h>
#include <Ethernet.h>
#include <PubSubClient.h>
#include <SoftwareSerial.h>

//DEFINE SENSORS
const int totalSensors = 4;
const int startSensorNum = 1;

SoftwareSerial sensor1(14, 6); //RX, TX
SoftwareSerial sensor2(15, 7); //RX, TX
SoftwareSerial sensor3(16, 8); //RX, TX
SoftwareSerial sensor4(17, 9); //RX, TX

SoftwareSerial* sensors[] = {&sensor1, &sensor2, &sensor3, &sensor4};

//DEFINE NETWORK
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
IPAddress ip(192, 168, XXX, XXX); //comment this line if you are using DHCP
IPAddress server(192, 168, XXX, XXX);
EthernetClient ethClient;

void setup() {

  Serial.begin(9600);

  //Initialize Sensors
  sensor1.begin(9600);
  sensor2.begin(9600);
  sensor3.begin(9600);
  sensor4.begin(9600);

  // Ethernet.begin(mac); //Use this if you want to use DHCP
  Ethernet.begin(mac, ip); //Use this if you want a static iP.
  delay(1500); //Used to allow time for the network to connect.

}

void loop() {
  // put your main code here, to run repeatedly:

}

MQTT Client Setup

Now that the basic parameters are setup, the MQTT parameters can follow! Start by defining the ID for the client. I am using “GardenCenter” as that is where I am going to place my Arduino. Any name/ID can be used here!

Parameters

Second, set the base topic to publish sensor data to. As discussed in the Topics section above, each set of data has to be published to a unique topic in MQTT. The base topic will be set to the room/location this Arduino is in, and then each sensor or parameter can be appended to that string in a for loop rather than hard typing the full topic for each sensor. Just a cleaner and quicker way to write the code.

Now setup the PubSub Client. This takes 1 parameter that was previously setup, the ethernet client.

There are 2 more parameters to put in that will help with reconnecting the client without blocking the loop – the reconnect attempt (lastReconnectAttempt), and reconnect interval parameters (reconnectInterval). I have the reconnect interval set to every 5 seconds (5000 milliseconds), but this up to you.

The final parameter is a function that return a boolean value. This function tries to connect to the MQTT client, prints and error message if it cannot, and then returns the connection status. This is used in the main loop to make sure the client is always connected, and reconnect if it isn’t.

Code Blocks

Moving to the setup block, all that needs to be setup is the client. This takes 2 parameters, the PubSub client that was setup earlier, and the port of the server. The default port is 1883 for MQTT. If you don’t know what port to use, 1883 is your best bet.

The boolean function that was just created is now going to be used at the top of the main loop. This function checks first if the client is connected. If the client is not connected, a reconnect attempt will be made until the client connects. This uses the interval previously defined(reconnectInterval) to wait between reconnect attempts without blocking the loop. Once the client is connected, the loop runs to start sending data.

#include <SPI.h>
#include <Ethernet.h>
#include <PubSubClient.h>
#include <SoftwareSerial.h>

//DEFINE SENSORS
const int totalSensors = 4;
const int startSensorNum = 1;

SoftwareSerial sensor1(14, 6); //RX, TX
SoftwareSerial sensor2(15, 7); //RX, TX
SoftwareSerial sensor3(16, 8); //RX, TX
SoftwareSerial sensor4(17, 9); //RX, TX

SoftwareSerial* sensors[] = {&sensor1, &sensor2, &sensor3, &sensor4};

//DEFINE NETWORK
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
IPAddress ip(192, 168, XXX, XXX); //comment this line if you are using DHCP
IPAddress server(192, 168, XXX, XXX);
EthernetClient ethClient;

//MQTT Parameters
String topicBase = "location/room/sub-room/";
const char id[] = "GardenCenter";
PubSubClient client(ethClient);

long lastReconnectAttempt = 0;
const int reconnectInterval = 5000;

boolean reconnect() {
  if (client.connect("arduinoClient")) {
    Serial.println("Connecting to MQTT broker...");

  } else {

    Serial.print("failed, rc=");
    Serial.println(client.state());
  }
  return client.connected();
}

void setup() {

  Serial.begin(9600);

  //Initialize Sensors
  sensor1.begin(9600);
  sensor2.begin(9600);
  sensor3.begin(9600);
  sensor4.begin(9600);

  // Ethernet.begin(mac); //Use this if you want to use DHCP
  Ethernet.begin(mac, ip); //Use this if you want a static iP.
  delay(1500); //Used to allow time for the network to connect.

  client.setServer(server, 1883);

}

void loop() {

  if (!client.connected()) {
    Serial.println("Client Not Connected...");
    long now = millis();
    if (now - lastReconnectAttempt > reconnectInterval) {
      lastReconnectAttempt = now;
      // Attempt to reconnect
      if (reconnect()) {
        Serial.println("Connected!");
        lastReconnectAttempt = 0;
      }
    }
  } else {
    // Client connected
    client.loop();
  }
}

Reading and Publishing Sensor Data

This is where the magic happens. Even if you’re not using the same sensors I am, the process should be more or less the same. The Monk Makes plant sensors collect 3 data points, soil wetness, ambient temperature, and humidity.

In the main loop sensor data should be read at a defined interval. Two parameters will need to be defined to track this – an interval (I am using 30 seconds/30,000 milliseconds) and the previous millisecond time stamp . With those two parameters setup, the main loop can run a check with an if statement to see if enough time has passed to read the sensors. Using an if statement versus delaying 30,000ms prevents the loop from being blocked and the MQTT client from timing out the connection.

Now comes the main function used to read and report sensor data to MQTT. This data is going to be sent to MQTT as a JSON array with each item having 4 parameters – sensor number, temperature, humidity, and wetness. Typically I would use the ArduinoJSON library to create this file in a much cleaner manor, but there was not enough memory on the Uno to do this, so I wrote it manually as a string! If you’re using an Arduino Mega and want to have cleaner code, feel free to jump into the documentation on this library and implement it. It’s quite simple.

JSON configuration

To start the function, the String containing our JSON data needs to be created. It will start with a [ and will have array data appended to it throughout the function.

Next define the topic to which the data is going to be published as a String. This is simply going be the topicBase + “sensors” since all of our sensors are being sent in a single JSON file.

Next all of the sensors need to be looped through and have their data read. A for loop will do the trick for this. Inside the loop, the sensor number needs to be defined. This is calculated by taking the position in the loop (i) and adding the starting sensor number to it.

Since we are using an Array of SoftwareSerial devices, only one device can be listened to at a time. To do this, call the listen function for the current sensor, and then read the data from the sensors. The data can be printed to the console, and then appended into the JSON array. If this isn’t the last sensor in the array, add a comma to the JSON file to prepare for the next set of data. Finally, stop listening to the sensor.

After the for loop, the JSON string needs to be ended. To do this, append a ] to the end of it to finish the array. Now the string is complete and ready to publish to the MQTT client!

Publishing Sensor Data From Arduino to MQTT

To publish the JSON array to the MQTT client, all that is needed to call client.publish and pass along the sensorFullPubTopic string that we created earlier, and the sensorJSONOutput string. The trick is to append .c_str() to the String so that it will compile, as the function is looking for a Char array and not a String. One more thing to note, if you put the publish function in an if statement, you can print out if it successfully publishes or not. I chose to do this, but it is optional!

There are a handful of functions at the bottom of my code used to read the data from my sensors. These functions are unique to the sensors I am using, so you will likely need to change these depending on the sensors you are using. If you do have the same sensors as me, feel free to use these functions.

Final Code for Sending Sensor Data From Arduino to MQTT

#include <SPI.h>
#include <Ethernet.h>
#include <PubSubClient.h>
#include <SoftwareSerial.h>

//DEFINE SENSORS
const int totalSensors = 4;
const int startSensorNum = 1;

SoftwareSerial sensor1(14, 6); //RX, TX
SoftwareSerial sensor2(15, 7); //RX, TX
SoftwareSerial sensor3(16, 8); //RX, TX
SoftwareSerial sensor4(17, 9); //RX, TX

SoftwareSerial* sensors[] = {&sensor1, &sensor2, &sensor3, &sensor4};

//DEFINE NETWORK
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
IPAddress ip(192, 168, XXX, XXX); //comment this line if you are using DHCP
IPAddress server(192, 168, XXX, XXX);
EthernetClient ethClient;

//MQTT Parameters
String topicBase = "location/room/sub-room/";
const char id[] = "GardenCenter";
PubSubClient client(ethClient);

long lastReconnectAttempt = 0;
const int reconnectInterval = 5000;

boolean reconnect() {
  if (client.connect("arduinoClient")) {
    Serial.println("Connecting to MQTT broker...");

  } else {

    Serial.print("failed, rc=");
    Serial.println(client.state());
  }
  return client.connected();
}

//Loop Intervals
const long interval = 30000;
unsigned long previousMillis = 0;

void setup() {

  Serial.begin(9600);

  //Initialize Sensors
  sensor1.begin(9600);
  sensor2.begin(9600);
  sensor3.begin(9600);
  sensor4.begin(9600);

  // Ethernet.begin(mac); //Use this if you want to use DHCP
  Ethernet.begin(mac, ip); //Use this if you want a static iP.
  delay(1500); //Used to allow time for the network to connect.

  client.setServer(server, 1883);

}

void loop() {

  if (!client.connected()) {
    Serial.println("Client Not Connected...");
    long now = millis();
    if (now - lastReconnectAttempt > reconnectInterval) {
      lastReconnectAttempt = now;
      // Attempt to reconnect
      if (reconnect()) {
        Serial.println("Connected!");
        lastReconnectAttempt = 0;
      }
    }
  } else {
    // Client connected
    client.loop();

    unsigned long currentMillis = millis();

    if (currentMillis - previousMillis >= interval) {

      previousMillis = currentMillis;

      readSensors();

    }
  }
}

void readSensors() {

  String sensorJSONOutput = "[";

  String sensorFullPubTopic = topicBase + "sensors";

  for (int i = 0; i < totalSensors; i++) {
    
    int num = i + startSensorNum;

    sensors[i]->listen();
    Serial.print("Listening to sensor ");
    Serial.println(num);

    float temp = getTemp(i);
    float humid = getHumid(i);
    int wetness = getWetness(i);

    //Print Sensor Data
    Serial.print("Temperature: ");
    Serial.print(temp);
    Serial.print(" Humidity: ");
    Serial.print(humid);
    Serial.print(" Wetness: ");
    Serial.println(wetness);

    //Create JSON Data
    sensorJSONOutput += "{\"sensor\":";
    sensorJSONOutput += num;
    sensorJSONOutput += ",\"temp\":";
    sensorJSONOutput += temp;
    sensorJSONOutput += ",\"humid\":";
    sensorJSONOutput += humid;
    sensorJSONOutput += ",\"wettness\":";
    sensorJSONOutput += wetness;
    sensorJSONOutput += "}";

    if (num < totalSensors) {
      sensorJSONOutput += ",";
    }

    sensors[i]->stopListening();

  }

  sensorJSONOutput += "]";

  //Send Data to MQTT
  if (client.publish(sensorFullPubTopic.c_str(), sensorJSONOutput.c_str(), true)) {
    Serial.println("Successfully Published Sensor Data!");
  } else {
    Serial.println("Failed to Publish Sensor Data :(");
  }
  Serial.println(sensorJSONOutput);

}

//SENSOR FUNCTIONS
int getWetness(int i) {
  sensors[i]->print("w");
  while (! sensors[i]->read() == '=') {};
  return sensors[i]->parseInt();
}

float getTemp(int i) {
  sensors[i]->print("t");
  while (! sensors[i]->read() == '=') {};
  return sensors[i]->parseFloat();
}

float getHumid(int i) {

  sensors[i]->print("h");
  while (! sensors[i]->read() == '=') {};
  return sensors[i]->parseFloat();
}

If you enjoyed this tutorial, leave a comment and check out my other Arduino Posts!

JUMP TO TOP

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *