A project log for n00n - Real Time Music Sensor Streaming Protocol

MIDI is so outdated, welcome to the 20s and the 16-bit world !

yann-guidon-ygdesYann Guidon / YGDES 06/22/2023 at 20:310 Comments

The n00n protocol is destined to be used in a modern Max/MSP-style system that is totally software configurable, or like a VST-style virtual mesh that processes the sensor data instead of sounds.

n00n is designed to be easy to stream as digital audio (just don't listen to it !) so some tricks are possible. Many functions can be implemented at the sensor level or at any level downstream, recording taps can be placed at any place in the stream... The idea is that a performance can be "recorded" right at the sensor level, and then "replayed" exactly as is, but all the technical parameters can be readjusted later (a bit like what photographers do with "raw" files versus .jpeg files).

So physical interfaces can be as simple as pots (for example) directly tied to an ADC and a microcontroller or FPGA reads it continuously, with only minimal processing. The stream can then be turned into useful data with a DAW that manages "filters" to get the desired user response, for example from a keyboard:

  1. Scaling (requires calibration of min and max values to fit the range of 0 to 65535)
  2. Curve (could be linearisation, logarithm, exponential, sigmoid...)
  3. integration/low pass filtering (to reduce noise, smooth the data, increase accuracy)
  4. Acceleration, triggering, hysteresis, ADSR-like shaping...
  5. Eventually some time-dependent processing to add "echo", bouncing, delays etc. for added artistic effects...
  6. Interpolation/extrapolation to match the synthesiser sampling rate, to provide a smooth parameter.

Then the filtered value can be routed to a synthesis parameter input or filter banks, to be turned into continuous waves.

Due to all the possibilities offered by this type of filtering, the sensor could have a raw 8 or 10 bits range (filled by 0s in the LSB) but this is increased through the filtering and the oversampling. Hight sampling rate is preferred to absolute precision because precision can be recreated during the last step of interpolation, and the synthesizer units work more smoothly with high frequency updates of the parameters (finer steps create fewer artefacts and they are pushed higher in the spectrum). The user can tune the level of sensor smoothing to apply while mixing...

Sensor filtering could take place next to the sensor but this would also increase the complexity of the unit, because all the parameters need to be input somehow (with a fancy physical interface or through a dedicated protocol). Raw sensor output should still be available anyway (enabled through an option) if a keyboard provides a filter. The only critical parameter for a sensor (potentiometer, optical, capacitive, inductive ...) is the calibration : making sure min and max values are correctly set internally so the output values are properly scaled to the whole 16-bit range. This means that the sensor's controller must be able to perform at least efficient multiplication.