Prelude - Space within Spaces

A project log for Babel in Reverse

178 ESP32s under the Manhattan Bridge talking and singing in endangered languages to passers-by

Owen TruebloodOwen Trueblood 03/30/2023 at 02:111 Comment

Space within Spaces

Back in 2019 I worked on the software for another project with Joe - one of Joe’s installations called Space within Spaces. In that piece Joe had 180 lights hanging in a grid in the Juliana Curran Terian Design Center Atrium at Pratt Institute. They were to be animated based on particle collisions recorded by a muon detector. Each light was controlled by an ESP8266 over WiFi. The goal intent was to run the animations on a central computer connected to the detector and then stream them out to the array. Due to the large number of devices and the crowded airwaves on campus there was a technical challenge to overcome to get sufficient performance to achieve the intended effect. In the end I did find a solution, and that inspired the general architecture that I ended up implementing for Babel in Reverse.

Fast WiFi Streaming to 180 Lights

I joined the project near the end, after the hardware had been installed in the Atrium and the behavior was being programmed. There was already a first pass on the control software. But unfortunately it updated the array of lights extremely slowly - about one update every 10-20 seconds. This version of the software had each of the ESP8266s join a WiFi network and then run an HTTP server exposing a REST API to control the light. A Processing sketch generated the effects based on the incoming muon detections and then called the API of each light in turn to write an updated brightness value. Because several back-and-forths over the network were happening for each light in the array there was a lot of radio traffic and consequently high congestion that slowed communication.

We needed to reduce the number and size of messages being sent per frame. Because we were using HTTP over TCP each bulb update was taking 3 packets for the handshake to open the connection, one for the HTTP request, and then one for the reply. Multiplied by every bulb that meant at least 900 packets per frame, or 27,000 packets per second at 30 frames per second. Even setting aside the WiFi congestion issues that wasn’t workable, so I went looking for a way to reduce the number of packets being sent.

Each ESP8266 could control the brightness of the bulb in 1024 increments between off and full brightness. Since 1024 is 210 that means a bulb's brightness can be expressed in exactly 10 bits, which we would need to use 2 bytes to transmit. Multiplied by the 180 bulbs in the array we get 360 bytes for brightness, which is well under the 1500 byte MTU for the network. So we could make a single packet update the brightness for every bulb in the array at the same time. But how do we get that packet to every bulb without repeating it? And how does a bulb know what part of the packet to use to update its brightness?

The first question can be answered with a network facility called a "broadcast address." Basically these addresses are treated specially by a network and, if the network is configured to allow it, a packet with a broadcast address listed as its destination will be sent to every other client (read more in RFC 919). So we just take our update and put it it in a UDP packet that we send off to the broadcast address. The router sees our packet and retransmits it to every other client on the network, including all of our ESP8266s. Each bulb looks at the packet and pulls out the brightness that it should have and uses that value to update the duty cycle of the PWM signal controlling the LED brightness.

How does the bulb know what value in the packet to use? If each bulb had a unique identity and we had a map of where each of them were in the array then we could pick an arbitrary order for the values in the packet and then just flash the map into the firmware so each bulb could use it to look up its value in the right place. For the identity we can use the 32 bit chip ID in the ESP8226. In general it’s not guaranteed to be unique, but it was for the ESP8266s that we were using. I figured out where each bulb was by asking each to flash over the HTTP API and then asking for its ID once I found where it was. You can see the resulting map in the firmware here.

At this point the control problem is basically solved. We were able to update the array about 100 times per second. The control server was implemented in Processing so we could do the effects there. I also set it up to be controlled from TouchDesigner just for fun.

You can take a look at the implementation code in these git repos:

Towards Babel

This basic approach of having an entire array of devices receive individual packets that contain an update for the entire array via UDP broadcasts is simple, flexible, and powerful. As long as you don’t need the units to talk back and you can fit everything you want to send in one or a few MTU it will let you control many devices at a high rate over WiFi while being somewhat robust to airwave congestion.

I made many improvements and extensions for Babel in Reverse but the installation uses that same approach. I’ll elaborate on those details in a future post on the firmware.


matthieu goodwin wrote 03/30/2023 at 13:53 point

Very clever solutions! I have a young friend just exploring programming and this will be valuable for illustrating the problem solving process in engineering.

  Are you sure? yes | no