How we accelerated Modbus in our controller in a week

Initially, we did not consider it important, but over time it turned out that many of our customers needed Modbus, so we began to expand its support in our devices and solutions.

How we improved Modbus support

Our controllers on the ESP32 initially have a subsystem for working with the Modbus bus, but in the first implementations, the Modbus functionality was limited: the controller supported the simplest devices, and there were no ready-made templates. Although for the same relay modules with a small number of registers, the functionality was sufficient.

We described each Modbus device through registers and a response template in a small window, for example:

Response template (mask) — r1d1,r2d3,r2d1,r2,r1d1,rd2

outputs in order: voltage (1 register, 1 decimal place), current (2 registers, 3 decimal places), current power (2 registers, 1 decimal place), consumed energy (2 registers), network frequency (1 register, 1 decimal place), Power Factor (1 register, 2 decimal places)

Users could add support for their devices, but they needed to understand our syntax. All this was quite "painful" for users, they often added erroneous templates that did not work. Then they tortured our support to find the cause of the problems. We needed to make it better, so we switched to the second version of Modbus with ready-made templates.

In the second version of the Modbus subsystem for our controller, we already added ready-made templates to the firmware. Now users could not install a "crooked" template and ruin everything. The firmware only had the templates that we wrote ourselves. But if it was necessary to add support for a new template and device to the client controller, it was necessary to update the firmware.

List of ready-made templates, mainly for Wiren Board modules

Our user base continued to grow, so there were more requests from customers. Firstly, we wanted to give users the ability to add new templates to the controller without reflashing. Secondly, users needed the ability to write their own templates and add them to the controller.

This is how the third current version of the Modbus subsystem in our controller, which we called Modbus FS, came about. You can read more about it in the Lavritech documentation. We created a file system so that templates could be loaded as files without updating the entire firmware. Now the user can download the necessary templates from our repository and install them on the controller. You can write the templates yourself, but then you need to send them to us for verification, after which we will publish them.

Additional photos

Slow Modbus

We have figured out the templates and adding devices. But what about the response time? Users have long complained that Modbus devices respond slowly. For example, the light turned on a second after pressing the button in the app.

For systems with sensors, response time is not so important, there we poll devices once every ten seconds. In a smart home, the polling time can be reduced to one second. Less is already risky - the computing power of the ESP32 controller is not so great, it will be loaded with a large number of requests and will not have time to process other tasks. And the more devices on the bus, the more acute this problem is.

We tried the I²C bus for fast discrete input (buttons, switches). However, it is very short, as it is intended for communication of components on a printed circuit board. That is, through I²C it will not be possible to separate the same control modules and buttons over a long distance. In addition, devices from different vendors do not "get along" through I²C. Finally, users want a simple, universal and reliable way to connect external modules to the controller, and I²C is a "crutch", and a capricious one, you need to monitor the quality of the connection.

One of the implementations of I²C modules is the Wiren Board side modules, we have been using them for a long time. Yes, within the side modules everything works very quickly, but such a configuration is difficult to set up: it is inconvenient to expand, it requires constant control of the order of module connections, and it is also not suitable everywhere. We wanted better.

At the WBCE 2023 exhibition, we learned that Wiren Board has developed a Fast Modbus extension. The idea was beautiful, but would it work in practice? At the exhibition, we agreed with the engineers to send us the equipment under a warranty letter. After all, a demonstration is good, but you always want to touch it with your hands. We ordered a wide range of nomenclature, almost the entire spectrum of modules with Fast Modbus support. We laid out the devices on the programmer's table, assembled the bus, connected it to the PC via a UART adapter, and started driving and catching packets. What kind of "beast" is this? We quickly figured out how everything works. The "beast" turned out to be not scary at all. This is how the idea was born to add support for Fast Modbus to our Modbus FS subsystem, which we were just developing at that time. Was it difficult? At first, it seemed that there would be pain and tears, but in fact, no.

Adding Fast Modbus support to our controller

First, let's note two important points: to understand the operation of Fast Modbus, you need to have hardware on hand. Fortunately, Wiren Board can send them for free under a guarantee letter. And startup developers will not have to bear additional costs. By the way, our programmer did not even start to understand the operation of Fast Modbus until he received the "hardware" because he was lazy and had no time.

The second point is that everything is well described in the documentation. We tested the modules from Wiren Board in various ways for several days, watching the frequently blinking lights. It felt like the Modbus devices were right in the controller - the connection was (almost) instantaneous. We followed the documentation step by step and checked the operation of all functions. We figured out the priority of polls, event support on the bus, and bus scanning. Then came a complete understanding of how everything works. And if there is an understanding, then it's time to implement support in our Lavritech controllers.

We test our software environment first on a PC under Linux, and then compile the firmware. The programmer looked at the documentation, examples, and then wrote the code. The entire work was divided into four stages:

1. Subscription - the master sends commands to all devices to indicate which registers to monitor.

2. The master sends a broadcast request for new data to all devices.

3. Then, according to the algorithm, the devices transmit packets, we parse them.

4. The last stage is bus scanning via Fast Modbus. There were initially errors here, but everything was quickly resolved during debugging.

It took one of our programmers a week of evenings to add Fast Modbus, while he was solving other work tasks in parallel. So all the support can be done in one sprint. As a result, we added a bus device scanner, event support, changing device speed with a single request, etc. to our firmware.

In the code, all this looks like an additional thread that periodically requests data on the Modbus bus, then checks the next packets in a loop. Essentially, a thread with periodic requests that does not intersect with other data.

As a result, we set the polling interval to 100 ms, although the documentation indicates the possibility of setting it to 50 ms. But even 100 ms is almost instantaneous response.

To enable Fast Modbus in the controller, you need to have firmware with its support, after which you can activate the option with a checkbox in the web interface. The templates have a note: if you need to use Fast Modbus for any registers, set it to one. That is, the client can choose for which devices he needs Fast Modbus. If Fast Modbus is not needed, then communication with such devices and registers will proceed in the traditional way.

How did Fast Modbus affect the load on the controller? After all, the ESP32 has few resources. We were worried about this, but it was completely in vain. We even managed to reduce the load. How? We increased the polling time of regular devices to 10 s, and the polling of Fast Modbus required almost no additional resources. As a result, we even sped up working with I²C devices, as the controller's resources were freed up.

Conclusion

Initially, Modbus was not our main focus, but it turned out to be needed by clients, so we gradually improved it. When working on the third version of the Modbus subsystem in our controller, we learned about Fast Modbus. The idea seemed interesting, so we decided to add support.

To be honest, we expected it to be more difficult. But Wiren Board transferred all the complexity of Fast Modbus to the side of devices and firmware, so it didn't take much time. We added it literally in a week, and some time was spent on testing. As a result, we even managed to reduce the load on the controller, as slow devices do not need to be polled often, and those critical to delays work through Fast Modbus. The freed-up resources even allowed us to speed up work with I²C devices. We are satisfied. We hope our clients are too.

What do you think about Fast Modbus? Do you have your own implementation cases? Share in the comments.