An attempt to create a programming trainer that uses physical blocks, instead of using drag-and-drop on a computer screen. The aim is to create a tactile, tangible programming experience to make learning programming easy and fun.
In late 2018 while studying at Monash University, I started a project for the unit "FIT3146 - Emergent Technologies". This was a "physical programming language" using Arduinos as programming blocks. The proof of concept was a success, so now I am documenting what I did with the aim of taking this idea further...
The Tangible Programming project is a new way of teaching programming. Instead of learning to program by typing code, or by using drag and drop (a la Scratch), Tangible Programming is designed to be a tactile programming experience. Using physical modules which you plug together, you create a program just like building with Lego. By plugging the brightly coloured modules into each other you build up your program. The main control unit will show you which blocks are attached and when the entire program is connected and read in you can follow the progress as it executes as each module lights up as it is being executed.
While the current prototype only offers a few commands, more programming commands are planned, including support for strings, functions and even integration with Arduino sensors and modules.
Okay, so after talking to some friends at university, I've decided to call this project SPLaT. That stands for Simple Programming Language Trainer. I'm still not sold on the name to be honest, but if Scratch can survive, why not SPLaT?
I have also thought of another project that can reuse many of the same modules (or at least, very similar modules) which will mean the Tangible Programming moniker will still stand, but more on that later... ;^)
I still have a few issues to iron out, but I managed to find an OLED library that was small enough to fit on the ATTiny841 and got a small, 0.49" OLED display working on an output block. Now, as the program is running, the output block shows the current value of a variable. I even have enough spare memory to use two different fonts, which I was a pleasant surprise.
This has highlighted some issues with my power distribution. I really need to work out what is happening with the power. The current set up doesn't quite provide enough power to initialise the OLED on the block, but if I power up the block first with an external power supply, then add the block to the other blocks while it is still powered, then apply power to the master controller, it will work. I think that might be a little too hacky...
I do want to replace the Arduino Uno with a custom board, so maybe that should be the next thing I focus on. I can try to work out the power issues while creating the master controller's board. Or, do I really need a master controller at all? If I can stick a display on individual blocks, the need for a master controller to display the output is reduced. Something for me to think about...
I really should come up with a proper name for this project. Up until now I have mostly referred to this as "Tangible Programming" or "Programmable Blocks". Tangible Programming is closest to what the project is, but it isn't a particularly enticing name. When I wrote my report for the university assignment this came from, I used the name SPLaT (Simple Programming Language Trainer) but I'm not sure about that as a name.
Any ideas? Anyone? What would you call this project?
I am quite happy with my latest PCBs for this project (thanks JLCPCB!) although they do have some issues. The matte black boards look particularly sexy, although flux residue can be a bit of an issue if not cleaned up.
I thought it'd be nice to give you a brief tour of the board and talk about some of the things I will most likely be changing for the next revision. The next revision will be some time off, but I am building a wishlist of things to change/add.
A Brief Tour
So let's have a quick look at the current boards:
First of all, please ignore the missing Vin at the top left of the board - somehow I managed to delete it before I generated the Gerbers. 8^/
The board is designed to be somewhat flexible, with two incoming serial connections, and two outgoing serial connections. (Highlighted in yellow above.) Both of the ports at the top of the board are connected, as are the two ports at the bottom. This means I can choose to solder the 4-pin connectors in either position, which lets me create blocks that "indent" the code. This can be seen in my favourite demo program (which calculates the Fibonacci sequence) where the yellow LOOP block makes sure the loop contents are nicely indented and the END LOOP block un-indents the code.
At the right end of the board is a four pin port which is used to connect to other modules. Currently I am using this for value and expression blocks. The port uses I2C, so theoretically any I2C module could be plugged in. (With the appropriate code modifications of course.) That means it might be possible to plug an OLED display directly into a board, or even sensor modules such as temperature, light, or sound sensors, opening up a whole world of new possibilities.
The last port on the board is a programming header. (Circled in blue above) I built a pogo-pin adapter with a USB to FTDI module to program the blocks via this port, and it all seems to work fine.
Lastly, there is a white rectangle at the back of the board where I can label the board's function if required. At the moment I have been able to keep each colour to a a different programming construct and thinking about it, this label might not be needed. For example, yellow boards are being used for LOOP/END LOOP and I will build some yellow variable boards, but the placement of the input/output ports will be enough to distinguish variables from loops.
Other things to note about the boards: There is no power regulation on the board. This was not an oversight - it is by design. The boards draw their power from the input port and the master controller (currently an Arduino Uno, but soon to be a separate custom board) is used to handle the regulation of power. This has allowed me to reduce the parts count and make the boards a bit cheaper to make.
There are two indicator LEDs - one red and one green. The boards can use these to indicate their current execution states, or if they have an error.
And apart from that there is not much else to these boards. K.I.S.S.
Alas, it is not all smooth sailing however...
Flashing the Bootloader
Apart from the missing Vin label, there are a couple of other issues with the board. Starting with the biggie - I forgot to consider how I would flash the bootloader. Oops! While I have the FTDI header, that is not used when flashing the bootloader and setting fuses on the chip. I did manage to get it all working, but it is a bit of a bodge.
To flash a bootloader you need access to an ICSP port. On ATTiny (and ATMega) chips, that means connecting to VCC and Ground (got both of them on the serial ports and the I2C port), MISO, MOSI and SCK (those are TX Out, SCL on the I2C port and RX Out) and the RESET pin. There is a RESET on the FTDI header, but that is connected to the RESET pin via a capacitor, and the ICSP protocol doesn't like that idea. 8^(
I was able to flash the bootloader by connecting the the five broken out pins (basically the outgoing serial port...
When I built my first breadboard prototype for my university class, I had a problem where only the first eight blocks would be recognised. When I checked the voltage on each module, I realised that after about eight blocks, the voltage had dropped too low for the Arduino Pro Mini clones to effectively operate. It was actually at around the 11th or 12th module that the Pro Minis would not boot, but I figured that the 9th module didn't have enough power to send or receive serial communications. Luckily my showcase program (the Fibonacci Sequence) only needed eight modules, so I waved the issue off as a power problem and stuck to programs of eight steps or fewer.
With the move to ATTiny841-based blocks I was able to reduce the power requirements considerably, so I ought to be able to have more than eight blocks in a program. Unfortunately my testing showed otherwise. The power was getting through okay, so something else must be at fault here.
I did some testing by tweaking the code. Instead of sending a broadcast message to all modules, I sent an "identify yourself" query to a small handful of select blocks. That showed that I could communicate with the 10th, 11th, 12th blocks with no problems. But if I sent a broadcast query, only the first eight blocks would respond.
Maybe forwarding the broadcast message was failing after eight hops? That didn't seem right, but I through I would try to rule that out. My next test was to send queries to each block individually - i.e. not using a broadcast message. Here is where it started to get weird: Sending 13 ID queries in order (from 0 through 12) still only returned eight results, but the fourth block's response didn't appear. This seemed to indicate to me that timing was an issue, and the timing issue was most likely in the master control unit (MCU), rather than the nodes.
I decided to manually trace the code in the MCU and noticed that I was sending the messages and then clearing the screen in the setup() function and the loop() had no particularly intricate operations during the receipt of the messages. So I moved the code to clear the screen to immediately before the message(s) were sent and bingo! All 13 connected blocks responded and were displayed on screen.
So I can now create programs up to 13 blocks long - any more and I won't be able to display the full program on the tiny screen I am using! As an added bonus, switching the the '841 has reduced the power consumption such that I can power the whole system off a USB power bank. (The Pro Mini modules were too hungry for battery operation and I needed a wall wart to power even four or five modules, let alone eight or more.)
Now that I know what was causing the issue I will revisit the code when I have some spare time and try to make it even faster. I will also look into how I can display longer programs. I am quite relieved that I found this problem and that it was such a simple fix. My biggest fear was that there was a problem with the circuit board, or that there was a more fundamental error in my design.
Update (16 April 2019)
Today I was showing this project to my Honours supervisor while we discussed my Honours project for next semester. I want to do an Honours project based on the communication protocol I am using - i.e. are there any existing protocols that might be better, or am I on the right track with this protocol? I explained to him this issue and how I fixed it, but I still wasn't 100% certain why it was happening - just that it was a timing issue of some sort. As I said this to him it struck me what the underlying issue was...
When the master control unit (MCU) sends out the broadcast message and then starts the screen clear, the blocks receive the message and start replying immediately. The MCU's serial input buffer starts filling up with the incoming replies while the screen is being cleared. By the time the screen clear has completed and the MCU gets into the...
First up, an apology for the lack of updates since I received my last batch of PCBs. There have been a few things going on in my life that I am not willing to discuss here, but it has been a rather trying time. I have been focussing on my family and my studies (in that order) and all of my projects have taken somewhat of a backseat. I'm slowing getting back into the swing of things...
Secondly, another apology is due. I had recorded the unboxing of my PCBs and an initial investigation into the quality, but I am a doofus and the recording was all but unusable. The audio was ruined by a neighbour's intermittent, loud music and the video was affected by a big, clumsy doofus knocking the tripod. After trying to cut around the damage and re-record the voice over, I have decided to give up - it was taking me too long and I am not a videographer. I will try o chop what I have up and reuse some pieces if I can, but the whole unboxing experience was a non-event - sorry about that.
Some Colourful New PCBs...
As you may know, I recently received a batch of PCBs from JLCPCB, who kindly provided them free of charge. This was a great boon, because I am a (mature-age) student, currently living off student payments (Austudy) and as such I don't have a lot of disposable income. JLCPCB recently dropped the price of their coloured PCBs and I thought it would be neat to see how the colours worked for my program blocks. In my original prototype I used coloured breadboards to represent different programming constructs. For example, Red, Green, Blue and Yellow blocks were variables, White blocks were output blocks. Now that colourful PCBs are affordable, I could implement a similar colour scheme again instead of having all green PCBs.
I quite liked those colourful blocks, and have preserved a set of them for posterity. 8^)
At around the same time I had been experimenting with the ATTiny841 as a replacement for the Arduino Pro Mini clones in my blocks. The '841 is a neat little chip - it has two serial ports, which I can use for the incoming and outgoing connections, and 12 GPIO pins ('though I am using four of those for the serial ports). A bit of ugly breadboarding proved the chip was up to the task, so I decided to take the leap and use the chip in the next iteration of the design.
I somewhat cheekily reached out to JLCPCB via Twitter and their representative offered to pay for my next batch of boards, so I went about designing a new PCB layout using the ATTiny841 and placed my order for a small batch of five panels of each colour - Green, Blue, Red, White, Yellow and Black. The new boards would be quite sparse compared to the previous iteration - the '841 is much smaller than an entire Pro Mini and the reduced number of available GPIO pins (out of the 12 available, four are used for 2x serial, 2 for I2C and one for the reset pin, leaving 5 free) meant I had to drop the jumpers for the block id. The lack of block id jumpers means I have to hard-code each block's id, which is not that big a deal IMHO.
Unfortunately (for me) the '841 is only available in an SMD package, so I went the whole hog and used all SMD in the new design. I included an FTDI-compatible programming header on one end of the board so I could easily program the blocks when needed.
Examining the New PCBs
The new panels indeed looked quite sexy!
And breaking the boards apart leaves some very sexy, tiny modules!
And here is where I discovered my first mistake (thankfully a tiny one) - the Vin label on the first connector was missing. I don't know how that happened, but it is an easy fix.
A close inspection of the boards showed them to be very nice. The matte black boards in particular look gorgeous, but all the colours are quite nice. One complaint I have however is the white...
After JLCPCB dropped the extra charges for coloured soldermasks, I decided to order a batch of PCBs in several colours to allow for quick and easy identification of program blocks. I (somewhat cheekily) pondered aloud on Twitter whether someone might sponsor this order - I am currently a student and even though JLCPCB's boards are cheap, when ordering six different colours of two designs, it still adds up. Alice from JLCPCB replied and kindly offered to cover the cost of the boards (and shipping!). 8^)
After trying the new, ATTiny841-based design on a breadboard, I finalised my new PCB layout and sent off the Gerbers. A week later a courier was knocking on my door with a nice big parcel. I resisted the urge to tear into the box and waited until I was able to film the unboxing (I need to tidy up the footage before I post it) and get a good look at the new boards.
So far I am impressed. There are some slight silkscreen issues on a few of the boards, but overall the quality seems excellent. I will post a follow up with a detailed review of the boards and the results of a test build, but for now I'll leave you with two pictures of the boards...
The past week or so I have been exploring the use of the ATTiny841 for my programming blocks in place of the Arduino pro Mini clones I was using. There is nothing inherently wrong with the Pro Mini and they are cheap enough to use, however plopping a chip directly on the PCB will make for a more robust (and even cheaper) system. Brian Lough was experimenting with the '841 on his YouTube channel a few weeks ago and it looks to be an impressive little chip. It has 2 USARTs (which will be handy for the incoming and outgoing serial data for this project) and comes in a 14-pin package. SpenceKonde's ATTinyCore supports the chip and provides software I2C master support, although for now I am using an external library for the I2C. (I need to determine if I can change the default pins for the built-in soft I2C support - my initial attempts were not overly successful.)
I successfully breadboarded an ATTiny841-based block today and happily it seems to work fine. It looks a bit messy on the breadboard, but looks aren't everything. ;^)
Now that I have the breadboard prototype working, it is time to commit it to a PCB. Due to the relatively low GPIO pin count, I have decided to switch to a hardcoded block id, which simplifies the PCB design. I'm keeping the same form factor as the Pro Mini based design and the final product will look quite stark in comparison! Apart from the connectors, everything is now surface mount. This was not an easy decision for me, as I am through-hole through and through, but the '841 only comes in an SMT package and it just felt odd surrounding the SMT chip with a couple of THT capacitors and a handful of THT resistors and LEDs.
With luck I will be able to get these new PCBs in multiple colours so I can use the board colour to visually identify the block function. (NB: I envisage the final version of this product will be housed in coloured cases, with each case having a different shape , so the function of a block will be identified by colour and shape (and possibly texture?) so as to cater for colour-blind users. I also want to key the connectors based on block type if possible so it will be difficult (or impossible) to plug the "wrong" blocks in to each other...)
When I started this project, it was simply a university assignment. I had no thoughts of taking it any further than just enough to get a decent grade. However, as I started preparing prototypes and experimenting with different options I realised that there was some serious potential in this concept. This could form the basis of an interesting way of teaching kids how to program in a very tactile manner. I was lucky in a way, that early in the prototyping stage I realised that the mini breadboards I had were almost the perfect size for the modules. And by "almost the perfect size", I mean they were exactly one row of holes larger than I needed. (Notice the empty row of holes between the Pro Mini and the four in/out pins.)
The perfect sizing immediately introduced the idea of using colour to distinguish the programming blocks. The mini breadboards I had came in six different colours - yellow, blue, green, red, white and black.For my prototype I chose yellow, blue, green, and red as variables, white for output and black for "end program". (I also used slightly longer breadboards for loop/end loop as I needed some extra rows to "indent" the code blocks.)
I liked the idea of colour-coded blocks, although using distinct colour blocks for each variable is a little hinky, but it worked well enough for the prototype - I am now considering having a single variable assignment block that takes a dongle to indicate which variable it is modifying. When I made my first batch of PCBs for this project (one problem with the breadboard prototypes was the four pin male/female headers are too easy to rip out of the breadboard) I had to (temporarily) ditch the idea of colour-coded boards. Instead, I planned to eventually create colourful cases for the blocks. JLCPCB (who I used for the PCB fab) do offer coloured soldermasks, but the cost was prohibitive for my meagre budget - effectively doubling or tripling the cost of the PCBs just for choosing colours other than green. Instead I chose plain green and to further reduce costs I made my boards multi-purpose. The boards have a socket for an Arduino Pro Mini, two sets of input pins at the top so you can choose to "outdent" code (for example, an ELSE, or END LOOP block), a corresponding pair of output pins at the bottom to optionally "indent" code (LOOP or IF blocks), a set of block ID jumpers to set the block type, I2C output pins at the right and jumpers for any spare GPIO pins "just in case".
Adopting this approach has allowed me to drastically reduce the costs of the PCBs - one design can be used for almost any block type. However, while I only need a single PCB design for the blocks, I need ancillary modules for values, expressions and other block-type-specific functions. As I design later revisions of the PCBs, I am moving towards having distinct designs for specific types of block. Variable assignment, IF and LOOP blocks all require a variable and/or expression - for these blocks it might be best to build-in the initial value component and then allow the addition of an expression module to build up more complex expressions. I have started down this path and while I lose the "universality" of the blocks, it allows me to think about custom code for specific module types and offers some other advantages. I have a set of PCBs designed along these principles and I do think this is the right path to take moving forward. The block below has an integrated MCP23008 and headers to plug a value or variable dongle into. (I realised after I had soldered up this batch that I didn't need to MCP23008 - I could have used the broken out GPIO pins A0-A3 and D10-D13 instead - doh!)
My next step will be to remove the Pro Mini and use either an Atmega328PB (this variant has two UARTS!) or an ATTiny841 (again - TWO UARTS!) soldered directly on the board. Removing the universality (or limiting it to a degree) and using custom firmware for each block type (so I can remove the...
I mentioned earlier that this project started as a university assignment. For the assignment, I chose to implement a small subset of the language I had defined, namely: Variables; Output; and Loop/EndLoop. To simplify detecting the end of a program I also included an END_PROGRAM block.
For the variables, I used the four brightly coloured breadboards in the mini-breadboard pack I had. That gave me an easy way to refer to the variables - use the colour of their breadboard - giving me four variables, Blue, Red, Green and Yellow. The four variables defaulted to a value of 0 and could be assigned the value of another variable, a two-digit integer, or a simple expression comprising two operands and an operator.
Output was quite simple - it would output the value of a specified variable.
Loops could be programmed to repeat a set number of times.
For both Output and Loop, their input parameters were dongles I created to represent each of the four variables. These dongles could be plugged into the breadboard of the Output/Loop module and read via the relevant GPIO pins. As a shortcut, I allowed variables to assigned using the same system, avoiding the need for too many PCF8574-powered value blocks and expression blocks.
While it added complexity to the Arduino sketch, I decided to use a single codebase for all the programming blocks. Therefore, when a block is first powered up, the code checks for some jumpers on three GPIO pins to determine the block's type which is stored in a block_id variable. Then it checked to see if there were any I2C devices at addresses 0x20 and 0x21 - the addresses of the PCF8574 ICs. If only one device was found, that must be a single value module, if both addresses were found we have a full expression module. The block then enters its loop() loop, waiting for any incoming messages on both the hardware serial port and the software AltSoftSerial port.
The master controller (I used an Arduino Uno with a small OLED display for this) would send a broadcast message (WHO_IS_THERE) on the serial port asking any attached modules to identify themselves and then enters its own loop() to wait for any replies.
When a block receives a WHO_IS_THERE request, it will respond depending on its internal block_id:
Variable blocks determine their value by reading values via the attached PCF8574 IC (if any) or if no PCF8574 is detected they will read the value of a dongle plugged into the breadboard using GPIO pins. Having determined the current value of the variable (or its expression), a response is sent to the Master Controller containing the operands and operator of the expression, or a single value as appropriate.
Loop and Output blocks responded with their block_id and their associated value.
EndLoop and EndProgram blocks simply respond with their block_id.
After sending the WHO_IS_THERE command, the Master Controller waits for responses until it receives an EndProgram, storing the block_ids and their parameters in an array. Once the EndProgram has been detected, the Master Controller switches to execute program mode.
To execute the program, the Master Controller steps through the program array, interpreting each command as it goes, maintaining an internal representation fo the four variables. To indicate the progress of the execution, the Master Controller will instruct the currently executing block to display its green LED. All Output commands cause the respective variable's name and value to be displayed to the attached OLED display. Execution continues until the EndProgram block is reached, or an error is encountered. If an error is detected, the relevant block will be instructed to blink its red LED to let the programmer know where the problem is.
And that is pretty much all there is for now. The blocks are really only "dumb" placeholders and all the smarts lives in the Master Controller.
Since the successful completion of this initial prototype I have created some PCBs to build some more...