Need to write a proper description, but in short, the pipeline is Jenkins JSON API => Rasberry PI running a Python script => Rasberry PI SPI Interface => Arduino converting from SPI to WS2811 protocol => WS2811 led strip
As mentioned in previous log entry, the LPC1768 micro-controller weren't up for the task and I decided to switch to a Raspberry PI! Naturally there are pros and cons with this kind of solution, namely:
Software implementation will be simpler as it will be possible to use a high level language such as Python.
Iteration of the software will also be faster as it is just a matter of restarting a Python program, no need for flashing images to the micro processor, just SSH to the Raspberry PI.
One negative aspect is that stock Raspberry PI images gives no guaranties of realtime performance.
It is also an overkill solution, a Raspberry PI has way more power than needed, and why would you need an entire operating system when connecting to the internet and reflecting status on some leds.
With that the project was done, I wrote a quick and dirty Python script which connected to our Jenkins server, it polled the latest Job status and queue length. Bit banged the data to the WS2801 strip, one led for each job, and from the bottom of the strip I added one blue led for each 10 builds in the queue. The led color was mapped to reflect the status of the Jenkins job, green led for stable builds, orange led for unstable builds, and red for failing builds. There are a few other odd/uncommon job statuses, namely not-built and aborted, the color for these were. Ongoing builds were illustrated by flashing the led intensity.
All of this was done in two threads, one which polled the Jenkins server for current status, about every five seconds. The other handled updating of the led strip and animation of the flashing leds, this thread ran at 50hz for super smooth transitions.
As mentioned in the previous log entry, I had an one meter WS2801 led strip and a mbed LPC1768 micro-controller. The goal was to connect all of this to the network, poll the local Jenkins instance for job status and present it on the led strip, one LED for each job. As one soon may realise this will not provide very detailed information about what is running on the Jenkins instance or how it is doing. Instead it provides as quick and convenient overview of the status and motivates the team to keep the LEDs, and inevitably the jobs green. For example, suppose that you come to the office on Monday morning and you see that most of the LEDs are red, then you know that you will have a though start of the week. But on the other hand, if you see that it is all green, then you know that you can start coding right away. It also provides a good visual indicator for other teams whether the team is knee deep in build problems and do not want to be interrupted at this point to discuss some unrelated matter.
The goal can be broken down into several technical challenges that needs to be addressed in order to get it working:
How to communicate between WS2801 and the micro-controller
How to extract and interpret the information from the Jenkins server
How to connect the mbed LPC1768 to the network
How to write software which opens a TCP socket to the Jenkins server
The first three tasks was pretty straight forward. Unlike WS2811, which only has an data line and fixed timing protocol, the WS2801 has one data and one clock line. This means that the master controls the clock speed which makes life easier when it comes to implementation. The communication can simply be implemented either using built in hardware SPI or by bit-bang in software. I went with the later as it is quick and easy to implement and the amount of data to output is low.
Extracting and interpreting data from a Jenkins server is also straight forward. Jenkins provides a great XML and JSON API by default. Simply append /api/ to almost any Jenkins URL and you will see the output which is available. For example, go to the url /api/json?tree=jobs[name,lastBuild[result]]&pretty=true and you will get a list of all job names and the result for the last build of each job. There are two nice-to-know arguments in the request, the pretty argument which makes debugging easier. But, most important the tree argument, it allows for server side pruning of the response and is very powerful as it allows you to select exactly the data you ar interested in. By using this argument you limit the amount of data sent but also makes the life easier for the server as it doesn't have to spend as much time generating the query response.
With these three pieces of the puzzle done it might seem like we are close to done. Wrong, far from done! It was easy to bodge together some code which sent a TCP request to the Jenkins server, retrieved the data and parsed the information. Unfortunately I never got it to work reliably. Turns out that there is/was, at least back in 2013, a problem with the ethernet stack in the micro controller which causes it to crash after a few dozen of requests. Even modifying simple examples to make recurring network requests ended up crashing after running for a while.
In the end, after a lot of debugging I gave up on this approach and decided that the LPC1768 and mbed stack wasn't ready and up for the task! Instead, it was time for a second...
Throughout all my life I have been fascinated by light installations, and especially those involving LEDs. And, as the years has passed by I have made quite a few projects heavily oriented around leds in one way or another. Most of these projects has been failures, and are now residing, partially completed, in a box or drawer somewhere. However, just like a bland squirrel occasionally finds a nut, some of my projects actually have succeed and been appreciated by myself and other people. On this list we find the led clock, my simple temperature display, but most notably, and for which this project page is about, my Jenkins Led Strip Indicator.
It all started, back in 2011, when Sparkfun was selling some 1m 32 LED WS2801 strips for bargain price, $25 at the time. They were sold as ding and dent, there were some problems with the coating, but otherwise fully functional. Now, being the LED kind of guy, I had been drooling over these badboys quite a while but they where crazy expensive ($45), and it had been hard to justify buying one. So, when they were sold at almost half the price, I just couldn't help myself and had to buy one. But, once received, and the initial adrenaline kick had worn of, it quickly found its way into a box and got forgotten,. Although cool and flashy, I simply found no use for it.
Fast forward one year, and I was once again spending my limited student loan buying stuff from Sparkfun. Among the things I picked up was a mbed ARM LPC1768 micro-controller, without really having a purpose. And yet again, despite it having a really flashy online IDE and a freaking M3 ARM Cortex, it ended up in the box of long forgotten things.
Fast forward yet another year, I was done with my studies and I was spending my time as a full time employee writing code for a compiler. I had also spent the year introducing the concept of continuous integration (and continuous delivery) to the team and company. Previously building and testing was done locally by each developer before committing the code into the SVN repository. When you got test failures of build problems that you couldn't derive from your changes, then you sent an email to the other developers asking if anyone could reproduce the problem, or if they might have broken the code. Of course this wasn't ideal as a lot of time was spent figuring out if my changes had broken something or if it was someone else. So naturally some kind of continuous integration was needed which provided a green/red indication and central truth about the state of the code base.
After a short evaluation we picked Jenkins, I had previous experience and out of the bunch at the time it suited our needs. Now, I will not go into details about the benefits of a continuous integration system and which system to choose. All I'm going to say is that it accelerated the development and the developers could focus on writing code.
The number of jobs on the Jenkins instance grew quite rapidly and after a few months there were more than a hand ful of jobs. Things started to get out of control, there were too many jobs and it started to become a hassel to overview all the jobs through the Jenkins UI. So, we needed a fast and convenient status indicator. That was when I got the idea for a new LED project, a project which would connect to our Jenkins instance and poll the status of jobs and present in a simple and visible way. All of the sudden the seemingly pointless purchases a few years ago made sense. I quickly pulled out the old boxes with the WS2801 LED strip and mbed micro-controller and started connecting. That is how the first prototype was made...