06/24/2020 at 06:08 •
As promised, I will now discuss the parts of the platform in more detail – starting with the server.
The Communication Server
The main job of the server is to connect the prototype’s HTML and the ESP devices via MQTT and WebSockets. In a first iteration I used Node.js and Express to write it, resulting in about 600 lines of code. But since I am a UX designer and not a developer it's probably best when I don't write (a lot of) code.
The source code felt a bit crude and overly complex to me. It was hard for someone with little coding experience to get how the server worked. And that made maintaining and changing it hard for regular folks.
Luckily, I stumbled over an article describing Node-RED, a visual node-based programming language „for wiring together hardware devices, APIs and online services in new and interesting ways“. And it came pre-installed on the Raspberry Pi. So I gave it a try.
It actually took me a while to wrap my head around the paradigm of node-based programming. But after that, I realized that Node-RED was a brilliant choice.
And after using Node-Red for about three years, I am still in my honey moon with it. Node-RED is powerful, simple, often beautiful, and it’s level of abstraction is just right.
To show you what I mean, here is the /source-code/ of the server:
To give you a bit more context of what you are looking at: the purple nodes are MQTT nodes, the olive ones are for WebSockets, the the khaki one on the left is a node which evaluates the messages, and the one on the right stores a value in a variable.
The left flow listens for MQTT messages, determines its content and sends some to the HTML page via WebSockets. The right one does the reverse: it waits for a web socket message, stores it, and sends it out via MQTT.
As you can see, the server consists of six core nodes (that is counting all but the dark green ones, as they are output nodes I use for debugging). And that’s all there is to it.
Now, a quick detour to MQTT to help you to understand better how the server works…
MQTT is a topic-based protocol. You can subscribe to topics and you can send messages about them. All messages are handled by the MQTT broker. If you send a message about a topic, the broker forwards it to all interested parties (the subscribers). Topics have a hierarchy, using slashes. As an example, a topic for the IoT platform would be: iot/wemos01/button.
All topics of the IoT platform start with the iot prefix, followed by the element ID, and then followed by the (hardware) component. This is a topic the wemos01 would send, when the button connected to the ESP is pressed.
If you now look to the (purple) MQTT node on the left, you see that it listens to the topic: iot/#. The # symbol is a wildcard, thus the MQTT nodes has subscribed to all topics starting with iot/. It receives all IoT platform topics.
With Node-Red, I went from 600 lines of code to only six nodes, which is quite a ratio! The flow is simple and easy to understand. You can look at it and can see what it does. Brilliant, isn't it?
The Web Server
The one piece that is still missing, when we look at the architecture diagram, is the web server. But Node-RED comes for the rescue.
You design your Node-RED flows in your browser. This means, Node-RED has a built-in web server. In its settings you can define a folder for „static“ web pages. This is where we will place the prototype’s HTML and Node-RED will host them for us. Thus, Node-RED covers both the communication and the web server elegantly.
This concludes the third installment of the series. I hope you find the pace and the level of abstraction ok.
Stay tuned to see what’s next – as I actually haven’t decided what the next part is about. It’s either about Axure or about the hardware components and the ESPs.
We will see…
06/08/2020 at 15:41 •
Based on the before stated principles and requirements for the prototyping platform I came up with the following architecture:
On the left you see the web server where the HTML of the Prototyping tool will be stored. On the right, the different hardware components (sensors and actuators, e.g. buttons, rotary encoders, LEDs) are connected to ESP8266 microcontrollers. In broad terms, ESPs are similar to Arduinos but also offer WiFi connectivity for a low price. In between sits a server to broker and translate the information coming from both sides.
The ESPs will send MQTT messages when the user interacts with the hardware components, and receives them when the user interacts with the software part. The (static) HTML page will use the WebSockets protocol to send and receive messages.
These are the key components of the Prototyping platform. In the following, I will discuss individual components and provide additional details.
Source code and how-to will follow.
The Server of Servers and Wireless Connectivity
The two servers shown in the diagram above (the web server and the communication server) are running on a single computer (yes, a Raspberry Pi, as you might have guessed).
The Raspberry Pi is a cheap single board microcomputer. I am still using a Raspberry Pi 3, even though the version 4 is already available. The v3 is fast enough for his purpose. For the operating system I chose the default Raspberry Pi OS desktop version. (I’ll explain why I chose the desktop version later on, so bear with me.)
Since I wanted to have wireless connectivity to exchange messages and access the web server, I added an additional WiFi dongle to the Pi and configured it to provide an access point. Thus creating its own WiFi network.
Now the ESPs knows which WiFi to connect to and the IP address of the server is known as well. No need to open the web server or other parts of the solution up to the Internet. The core part of the solution needs no Internet connection and sets up it's own WiFi network.
But in case the Pi has an internet connection, I forward both the ethernet and the built-in WiFi connection to the access point connections to allow them access as well. This allows me for example to update the devices and access online services.
Note: the easiest way to access and setup the Raspberry Pi is to connect it via an ethernet cable to your computer and share your computer’s internet connection (Mac/Windows). A WiFi connection will drop when you follow the access point setup instructions.
Since the operating system and the data of the Pi are stored on a Micro SD card, it‘s really easy to make backups and to deploy last minute changes. And when you remove the card, all the intelligence of the hardware platform and the prototype are gone.
As you can see, the Pi has several jobs: it is the WiFi access point, it needs to provide a web server, a MQTT broker (which is needed to send/receive MQTT messages), and provides the communication server. An additional admin interface for the communication server would also be nice.
The Pi should share the web server’s folder on the network to allow for an easy way to upload the HTML prototypes. It should also be accessible remotely, to edit and to configure it, if needed.
In the next part I will start describing the different components in more detail.
06/02/2020 at 22:09 •
Vision & Guiding Principles
These requirements did not only help me make design decisions but also will provide some structure for the notes to come...