Back in 2016 one of my projects on Hackaday won me a 32x32 RGB LED matrix from Adafruit. At the time I didn't quite know what to do with it. It came with a Raspberry Pi shield to control it, however I thought it would be overkill to dedicate a whole Raspberry Pi for such a simple task.
Fast forward to now, the ESP32 Arduino support has come a long way. I was looking to make a project using Arduino IDE, Node-RED and MQTT (to add to my ever growing home-made IoT devices) and learn to do some more "complex" things on the ESP32. For the hardware this project technically only needs a LED matrix, ESP32 breakout, prototyping cables and a power supply.
I wanted to use my Raspberry Pi-based Node-RED server and MQTT to generate and stream the display, the reason being that it's much easier and faster to edit and add new animations on the server side, also more memory is available and processing is faster.
Recently, while moving and getting some furniture from IKEA, I found the perfect frame to host the RGB matrix, the RIBBA frame:
The matrix is 190.5mm x 190.5mm x 14mm, the frame is 230mm x 230mm x 45mm, plenty of space for the matrix and the electronics. The frame has a white cardboard insert that can be cut to the matrix size:
However the matrix will not be centred...
So I 3D printed some corners to center the matrix in the frame and push it against the front face:
The electronics are not looking very professional though, I need to rethink that, wires are pretty unreliable...
Writing the ESP32 firmware, with all its timing critical constraints, was only half of the job, the actual image generation happens on an MQTT server (hosted on a Raspberry Pi). I've made one of my most complex flows:
It does the following things:
Fetch the data from different sources (weather data, public transport, Bitcoin value)
Parse the data according to selected display
Evaluate the screen update rate (the ESP32 sends back a "heartbeat" message every time it is updated)
Ping the matrix to make sure it is present
Send the data to MQTT only if the pixels have changed and the matrix has been successfully pinged
Apply a "brightness filter" to set the pixel intensity level according to the user settings
Compress the data to a binary stream for efficient transport (as efficient as it can be made with this framework...)
Update the Node-RED user interface
In the user interface one can:
See the preview of what is currently displayed
Select the matrix display
Set the LED brightness
Change the display every minute (selects the next one in the list)
Turn off the LED matrix (send a black screen and then stop sending updates)
Having highly limiting restrictions is a fun way to practice problem solving and creativity. In the case of the 32x32 pixel matrix I wanted to have lots of information on the display, but were challenged by the resolution. Despite this limitation it was possible to achieve very interesting things.
This project being an Internet-of-Things device I wanted to fetch interesting and relevant information off the internet and local sensors and display them.
Super tiny font
I searched for a small font to display human-readable letters to convey information. I found the TomThumb font which squeezed latin characters in a 5x3 area, if you think about it it's pretty amazing how our brain can distinguish them at such low resolution (context helps a lot though):
I had to adapt some letters, as not all of them perfectly fitted the 5x3 area (g, j, p, q, y). This amazing font allowed 32/4=8 characters per line and 32/6=5 lines of text:
The first thing that I wanted to display was time. My ultimate goal was to replace my dad's wall clock:
Here are the animations I came up with:
This one was pretty easy, simply evaluate the angle from the hours, minutes and seconds and draw lines from the center to the edges. I liked the way the hands extend up to the very edges of the matrix. One can tell which hand is which by the intensity of the hands.
I also tried to use red, green and blue colours to distinguish the hands, but it was atrocious.
The digital clock only took the topmost line, so I added some more information on the next 4 lines: date, inside temperature, inside humidity and outside temperature which I got from openweathermap and update every 10 minutes.
The temperature colour is blue under 10C, cyan between 10-20C, green between 20-25C, yellow between 25-30C and red above 30C. The humidity is green between 40-60%, yellow between 20-40% and 60-80% and red otherwise.
This one was for the sake of the exercise, completely useless for its function as it's not easily human-readable. The numbers are defined by 6 bits, so maximum value of 2^6-1=63. The most significant bit was on the left side, first line is hours, then minutes and then seconds.
This example shows a time of 16+2:16+8+4+1:32+16+4+1: 18:29:53.
Reading the time and getting your head around the powers of two usually meant that by the time you understood the time it had already changed.
English word clock
Since there was enough space I wanted a clock that spells out the time in plain English. This is called a "word clock", there are many similar projects on Hackaday.io. Very human-friendly.
I tried to use only Katakana to tell the time, but there wasn't enough space as there was a hard limit of 4 characters per line and 4 lines with this font (16 symbols total). Also using only Katakana characters didn't look very nice.
Using Kanji fixed that and I must say I like it a lot:
今は午後六時二十六分です means "it is now 6:26 in the afternoon", although I doubt my dad can read it.
This animation was just a "why not" idea: a Bitcoin ticker that updates itself every minute. The data is taken from Blockchain ticker...
I had some ESP8266-12E's, but there weren't enough GPIO pins (11) to drive the LED matrix (13 needed). I got one of these standard ESP32 development boards off of eBay.
This board offered 25 GPIO pins, plenty enough and then some. At this point I didn't want to develop my own breakout for the ESP32 chip, that was not the goal. This board had everything I needed: the required passives (resistances and capacitors), LEDs, buttons, USB to UART converter and the 5V to 3.3V converter.
The first step was to look around if somebody had already made a driver for this particular chip and peripheral, I found that VGottselig had made exactly that. It was really great to start with something so advanced. However, this library did miss out on some features I wanted: it was hardcoded for 64x32 matrix, it used the Adafruit GFX library which I didn't need as I intended to generate the graphics remotely, it was not prepared for MQTT... so I used it as an inspiration for my own.
The things I mostly took from there were the way the GPIO were defined (much faster than digitalWrite() function), the way the display was updated with a timer/interrupt and the pixel light intensity control.
I went for the PubSubClient MQTT library for the MQTT part, this library limited a MQTT message to 128 bytes, including the overhead (topic name and whatnot). The LED screen had 32x32=1024 pixels, every pixel needed 3 colours and I wanted 4-bit colour depth (4 bits per pixel colour), that meat 32*32*3*4/8=1536 bytes/message, when rounded up to 2 bytes per pixel: 32*32*2=2048 bytes/message. This wouldn't work, even when I increased the hardcoded limit in the PubSubClient library (the ESP32 crashed).
Fortunately PubSubClient also allowed data streaming. It took some time to figure out how to use this feature, it felt like it was made exactly for this type of use case. Data was simply read into a buffer byte by byte as it came. This technically allowed for unlimited message size (within the ESP32 memory space of course).
I also got into trying to understand how things were organised on the ESP32 execution-wise, as the display update rate had to be really high to avoid visible flicker, but I also needed to dedicate some CPU time for MQTT message reception. The ESP32 uses FreeRTOS, which allows for concurrent task execution. It also has two identical CPUs inside, which share all the memory, this was a really nice feature: I could dedicate one CPU for display and the other for data reception and other tasks (WiFi stuff). Having shared memory meant I could simply write to the display buffer and the screen would update itself as data came in.
Having done the above I saw that when the screen was updating it didn't do it in one go, I could see the screen updating itself as the bytes came over (slow transfer).
This was no good, it didn't look right to have "scan lines" coming in, this was an issue with data transfer latency, but there was not much that could have been done about it. I measured about 4Hz maximum update rate. So I implemented double buffering: data was coming to a "back buffer" while the display was using a "display buffer" or "front buffer", once all data had been transferred the buffers would switch and the screen would be updated all at once. No more flicker!
The video is in slow motion and in two parts: with single buffer and double buffer. The camera picked up on the refresh rate of the screen, but that's not the important part here as it's invisible to the human eye. The difference is that in the first part, when switching colour, the pixels change gradually from top to bottom, as the incoming data is slower than the screen update rate. In the second part a "background buffer" is filled instead and then the back and front buffers are switched, giving the effect of instant screen update.
I looked if there was anything done on the software side for something less powerful than a Raspberry Pi. Some hardware is unfortunately very finicky and won't even work and lower clock speeds (WS2812 LED's, OV7670 camera...). Fortunately Adafruit had libraries and examples for Arduinos, unfortunately though the only Arduinos I had on hand were Leonardos (ATmega32U4 based), which were explicitly unsupported for various reasons.
The 32x32 RGB LED matrix is updated row-by-row, over 16 rows, upper and lower part at the same time. There are 6 pins for the color, so 3 for the upper half and 3 for the lower. 4 pins select one of the 16 rows (both for upper and lower matrices) and a clock signal that clocks in the pixel values. Then there are some other pins whose function I haven't quite understood...
As my ESP32 hadn't arrived yet I started with the Adafruit's library as an example and wrote a program to drive the matrix from an Arduino Leonardo. The reason they did not support this particular model was because they wanted to remain compatible on a certain number of other models, however the way the Leonardo pins are mapped to the Arduino pinout made it awkward to accommodate the library in a way that would remain maintainable. They did some really clever tricks to speed-up draw time to avoid visible flickering, such as using automatic pointer address increase in assembly (this is cycle counting business right there!).
I really liked the way Adafruit "parallel-banged" the GPIO directly from memory, basically the upper and lower pixel RGB values (6 bits) were stored on a single byte which was directly applied on an entire GPIO port array. This meant a simple loop could just sequentially bang the contents of a 32*32/2=512 byte array to a port without any conditionals or checks, considerably speeding up the screen update rate compared to, say, an RGB object that would define 3 bits per pixel to which the compiler would add all sorts of operations and jumps...