Rebuilding the compilation server

A project log for Barnabas Blocks Upgrade - BIG 2021

Webserial API webtool to upload Arduino code.

DanteDante 08/16/2021 at 21:540 Comments

In summary: I've rebuilt the API used to compile Arduino sketches. The old version would not adapt easily to our needs.

Several weeks ago, when I was doing digging on why the minified Blockly generators did not match the un-minified version, I stumbled on Chromeduino, a Chrome app for Arduino coding. It includes a web server where users can send code to have it compiled. In the source code for the web server, I noticed a /version endpoint with a rather specific output.

Barnabas has a code compiler server as well, with the same specific version endpoint. It turns out runs this API. Barnabas Blocks uses this site to convert C code generated by Blockly to the machine (binary) code sent to a microcontroller.

However, Chromeduino's compiler server has... problems. To name a few:

There exists an entirely new version of Arduino IDE called arduino-cli that exists to fulfill this need - other programs using Arduino. Instead of doing this, Chromeduino "hacks into" Arduino IDE, which is... problematic:

  1. First, it works by downloading a copy of Arduino IDE (despite never needing to open the actual graphical window) and running a utility program inside it which is not meant to be used by end users (it's more of a middle-man between Arduino IDE's compile button and the actual compiler program).
  2. Since this program is a middleman, it does not try to detect custom libraries, boards, or anything - it looks in and only in the list of places it is told to look (i.e. a list basically hard-coded into Arduino IDE). Specifically, the Chromeduino web server only allows libraries from its own copy of Arduino IDE.
  3. Content like boards and libraries not built into Arduino IDE are installed in another folder outside of Arduino IDE (~/Arduino on *nix and Mac and $HOME\Documents\Arduino on Windows). Chromeduino never looks at these, so it can't support custom boards or libraries.
  4. Instead of asking Arduino IDE to find libraries and boards, Chromeduino tries to do it all - looking for everything in one hardcoded location. Its practices in doing so are downright negligent.

Because of all of this, I decided to reimplement the compiler server using arduino-cli to support custom boards, libraries, and the like. It completely replicates the behavior of the Chromeduino server (meaning no changes are required of Barnabas Blocks in terms of changing how it sends C code and receives code back from a server). The result is currently under testing and will evolve as its use case evolves.

The project is licensed under GPL3+ and its code is viewable here.

Attached is a demonstration I made of compiling a short Arduino program which waits until a button on pin 2 is pressed and prints "press!" to serial.