Web server heap debugging

A project log for The Clock Awakens

Using LEDs and an ESP8266 to make a smart, futuristic Star Wars clock

Alex Cordonnier 01/03/2016 at 02:010 Comments

tl;dr: ESP8266WebServer fails to serve any static files when the heap is less than a few thousand bytes. 3,500 bytes definitely fails, and 35,000 definitely works. I didn't test to find the exact turning point.

I spent the past couple of days debugging an issue with the web server. Previously, custom request handlers such as /debug/reset, which restarts the ESP, would always work. However, statically served pages would only work intermittently. I could load a few pages, then would only get an ERR_CONTENT_LENGTH_MISMATCH for a few minutes, and then it would work again for a few more requests.

This seemed like a memory issue, so I printed the amount of heap space when it changed in loop(). I was consistently seeing around 3,500 bytes, which would dip to around 2,800 bytes whenever I made a request. It didn't look like it was running out of heap space.

First, I tried using a flat directory structure instead of using subfolders, but that didn't help. Then I thought it was just too big of a file, but it didn't work for tiny files either. I tried using curl in verbose mode, and I discovered that it was sending the headers correctly but simply didn't send any file content before closing the connection.

I tried implementing my own serveStatic method that would read the file and transmit it line by line, but it would only transmit part of the file before closing the connection.

Finally, I tried removing everything but the WebServer class, and the problem disappeared. I also had 10x the heap space, so something was up. I added classes back in one by one, and the offender was a header file called timezones.h. It had 418 timezone strings in an array so that they could be used with the timezone API.

I looked into using PROGMEM to store the array, but I kept getting strange compiler errors that I couldn't figure out how to fix. So I ended up just cutting down the number of timezones to Eastern, Central, Mountain, and Pacific. This does somewhat reduce the worldwide usability of the software, although the auto-detect feature will still work. Another potential solution is to store the list as a file, but reading the file would be O(n) instead of O(1) for accessing an array.