(Original post date - 04/01/16)

If you read my last log you would have seen that I started my project by prototyping ideas for the digitisation of the pianos key mechanism. Well, after spending the best part of a week attached to my soldering iron and multimeter I've finally completed most of the electronics and software for the final implementation of the key mech. The implementation for detecting key presses and releases has changed considerably since my initial design, and I've still got a bit of software tweaking to do, however I've now got a fairly reliable working key mechanism which is fully polyphonic, velocity-sensitive, and polyphonic pressure-sensitive.

Hardware

Components used:

• Velostat
• 10k resistor (x 18)
• 4051 multiplexier (x 2)
• Arduino Pro Mini (3.3V version)

Sensors

As per my prototyping, the sensor I have used to detect a press for each key is Velostat - a flexible conductive material that reduces resistance as pressure is applied to it. It is very similar to off-the-shelf pressure or bend sensors, but it has a number of advantages which have been useful for me:

1. It's a lot cheaper
2. It's more flexible
3. It can be cut to any shape or size
4. There are no set connection terminals

Using a pressure sensitive material was useful as it has allowed me to implement velocity sensitivity and polyphonic pressure sensitive is a fairly simple way.

Due to using Velostat, the circuit used for each key of the piano is essentially the same as that for an FSR sensor connected to a microcontroller, where one side of the sensor is attached to power, and the other to a pull-down resistor to ground as well as to an analogue input:

An FSR sensor attached to an Arduino. Source: https://learn.adafruit.com/force-sensitive-resistor-fsr/using-an-fsr

A small piece of Velostat has been attached to the underside of each key, and when a key is pressed it causes the Velostat to touch two contact points which closes a circuit (in the same way that a push button/switch works) and sends a pressure value to a dedicated analogue input for the pressed key. Since the prototyping stage I have rewired the keyboard so that the wires are soldered away from the area that the Velostat joins the circuit, allowing the contact area to be more even and allow greater touch sensitivity of the keys.

The Velostat attached to foam on the underside of each key, which connects two contact points going to power and ground/input when the key is pressed

Microcontroller

Even though it will be the BeagleBone Black microcontroller that uses the values from the keyboard to trigger and modify sound, I decided not to connect the sensors directly to the BBB but to use a second mirocontroller in between that communicates with the BBB via one if its serial interfaces. I did this for the following reasons:

1. Splitting tasks - The main job for the BBB in this project is to run a sound synthesis engine which is going to be time critical, so I don't want it to be doing any extra tasks that could slow it down. Also the scanning of the pianos 18 keys needs to be done as fast as possible so that the keys trigger sound as soon as they are pressed, so using a dedicated microcontroller for this task would be preferable.
2. More Modular - Connecting a microcontroller to the BBB rather than connecting 18 sensors directly requires a lot less connections and wires to the BBB, which makes it easier to remove the key mech or BBB from the piano if desired.

I chose to use an Arduino Pro Mini microcontroller for the key scanner as it is a small, cheap microcontroller that I personally have had a lot of experience with. I'm using the 3.3V version (rather than the 5V version) as I believe that the serial ports on the BBB run at 3.3V which would not be able to communicate with a 5V microcontroller without using a logic level shifter. As the Pro Mini only has 8 analogue pins I've had to use two 4051 multiplexers to connect all 18 sensors to the Arduino; the first 8 keys are connected to 1 multiplexer, the last 8 keys are connected to a second multiplexer, with the middle 2 keys connected directly to the Arduino. This has meant I'm only using a total of 4 analogue inputs, as well as 6 digital outputs to control the multiplexers. Finally the serial transmit (TX) pin is used to send serial messages to the BBB.

The Complete Circuit

The completed circuit for the key mech has been developed using stripboards which have been screwed to the base of the piano, as well a copper tape used as the power line to each key as well as for the contacts for the 'switch' below each key. Below is a breadboard diagram of the circuit I have built, followed by some photos of the circuit. 

Breadboard diagram of the completed circuit, where the Velostat sensors have been replaced by FSRs

Software

As mentioned above, all the processing of key interaction is handled using an Arduino microcontroller, so the only software required for the key mech is a single Arduino sketch that needs to handle four main things - processing key presses/releases, processing key velocity, processing key pressure, and sending these three attributes to the BBB as serial-based messages.

I have created a GitHub repository to host all my code and schematics/diagrams for this project. To see the current state of the key scanner code click here.

Processing Key Presses/Releases

Key presses are detected by reading an initial analogue input value of above 0, which happens when force is applied to one of the Velostat sensors which causes the keys circuit to be closed, whilst the key is currently flagged as 'off'. This will then trigger a note-on message for the pressed key as soon as we have generated a key velocity value (see below). Key releases are detected by reading an analogue value of 0 whilst the key is current flagged as 'on', which happens when one of the Velostat sensors is removed and breaks/opens the keys circuit. This will then trigger a note-off message for the released key.

Processing Key Velocity

Key velocity is the attribute of how soft or hard a key is pressed, and is most commonly mapped to the volume/gain of the triggered sound though can usually also be mapped to other sound parameters. My implementation of velocity sensitivity in the toy piano synth involves starting a timer when we get an initial key press, and then after 10ms it uses the highest sensor analogue value that was received to work out a velocity value.

For example, if a key is pressed lightly only a small force is applied to the Velostat sensor, causing the analogue input value reached within 10ms to be fairly small, giving us a small velocity value. On the other hand, if a key is pressed harder a greater force is applied to the Velostat sensor, causing the analogue input value reached to be larger, giving is a greater velocity value.

Once the key velocity has been generated it then sends a note-on message for the pressed key. I am using a time value of 10ms here as it has been found that the minimum latency the ear can detect is 11ms, so providing the key press triggers a sound before this time there will be no apparently delay.

Processing Key Pressure

Some synthesisers and MIDI keyboards have an attribute called 'Aftertouch'  - the amount of pressure/force that is applied to a key after is has been initially pressed, which can then be mapped to modulate a number of sound parameters. There are two types of Aftertouch - channel aftertouch (also known as Channel Pressure) and polyphonic aftertouch (also known as Poly Pressure), where channel aftertouch modulates every note of the same channel or keyboard at the same time, and polyphonic aftertouch modulates each note individually. Polyphonic aftertouch offers much better expression and modulation possibilities, however unfortunately it is quite rare among consumer keyboards and instruments as it is both expensive and more complex to implement.

As each key in my toy piano synth has its own pressure sensor, in this case I am able to implement polyphonic pressure sensitivity in quite a simple way. Currently it is implemented so that once a key has been pressed and a note-on message with a velocity value has been sent, any further changes to the force applied to the key while being held down that are greater than the initial force get converted to pressure messages for that key. However this implementation only works for softer key presses, as a hard press won't leave any scope for applying any more force that can be converted to an analogue signal, so I need to rethink how I can allow pressure sensitivity for all velocity values.

Serial Message Format

Being an electronic musical instrument, the most obvious message format to use here is MIDI. There are three types of MIDI messages that the Arduino will send to the BBB over serial, which are all three-byte messages that start with a status byte (a value greater than 127), which contains the message type and MIDI channel values; followed by 2 data bytes (which each have a range of 0 to 127):

• Note-on message - [144 + MIDI channel] [note/key number] [velocity value]
• Note-off message - [128 + MIDI channel] [note/key number] [0]
• Polyphonic aftertouch message - [160 + MIDI channel] [note/key number] [pressure value]

Example Video

Here is a demonstration video of the key mechanism in action. For this demo I have connected the Arduino Pro Mini to an Arduino Mega (through serial) which has been modified with HIDUINO so that it can be recognised as a USB-MIDI controller, sending MIDI messages to Logic Pro. Sending the messages from the keyboard to external MIDI devices will eventually be one of the tasks for the BBB.

Changes From Initial Design

The main part of the key mech implementation that has changed since the initial design is the removal of the set of switches on the back of each key, which were going to be used to generate velocity values by timing how long it takes for each key to be fully pressed; a method used by most consumer keyboard instruments for implementing velocity sensitivity. However after realising I could use the Velostat pressure values to work out velocity in a similar kind of way I decided not to add this set of switches, as it would double the size of the circuit and the amount of components. Also it would mean the Arduino would have to scan double the amount of inputs, possibly slowing down the key scanning task.

Therefore, contrary to my beliefs when devising the original design for this key mech and writing my last blog post, using sensors similar to FSRs has proven to be the easiest and most effective solution, rather than using a dual set of switches like consumer keyboards use.

Next Steps

Most of the key mechanism is now complete, and there are only a few things left to do on this part of the project:

1. Connect the Arduino Pro Mini to the BBB to provide serial transmission, as well as to power the Arduino from the BBB.
2. Some of the keys are a lot less sensitive than others, most probably due to the switch contact area not being even, which needs fixing.
3. A couple of the Velostat sensors can't reach high pressure values, causing the keys to have decreased velocity and pressure sensitivity, which also needs fixing. I'm not yet entirely sure why this is, but it could be down to the position or size of the sensor.
4. Each Velostat sensor produces slightly different results, causing the velocity and pressure sensitivity of each key to be slightly different. This could be addressed and calibrated in the software by defining maximum pressure values that each key/sensor can send.
5. Improve the polyphonic aftertouch algorithm so that pressure sensitivity works better when keys are pressed more forcefully.