Packet types

A project log for n00n - Real Time Music Sensor Streaming Protocol

MIDI is so outdated, welcome to the 20s and the 16-bit world !

yann-guidon-ygdesYann Guidon / YGDES 12/15/2022 at 20:320 Comments

Here I will try to list all the types of packets I can think of, to allocate numerical identifiers.

Each data source, or device, can generate any type of packet. A device can generate "keyboard" and "knobs"  packets, but can only have one set thereof. So if your device has 2 keyboards, you have to emulate 2 separate devices. Note : after being suggested the mixing tables, it seems I'll have to create structured packets with sub-fields, a "composite packet" but since it is not yet defined, the former will be simpler at first.

There are 16 bits in the header for the type and flags fields so each could take one byte at first glance, though there is no strict boundary. Flags could supplement the type and implement subtypes, for example, or an important type could use fewer bits and allocate more flags.

The 2 LSB are not reserved but that's where the compression flags are located :

Compression is optional and provided to save bandwidth so it might be ignored by some data sinks. That's why the source should send raw data once per second. Furthermore : compression does not work all the time and could eventually expand data, so raw transmission is always possible as a failsafe.

There are 4 big classes of packet types, using the 2 MSB :

As usual, if a packet type is not recognised or understood, it is ignored.


The Instrument class :

Type 0 is invalid (just in case).

  1. Knobs : just a collection of 16-bit values that could represent potentiometers, buttons, slide pots, ribbons, whatever : it's the most generic type.
    Format : 16-bit size prefix, followed by as many 16-bit unsigned values (that may be compressed, see the compression flags).
    The data sink is in charge of "patching"/associating each value with a meaning or function.
  2. Keys : a string of 16-bit values representing the key's absolute position for all the keyboard.
    Format : 1 byte of length prefix (the number of keys),
     1 byte of offset (the position of the lowest note/key)
    Followed by as many 16-bit unsigned values (that may be compressed, see the compression flags).
    Note : no other flags than the 2 compression LSB are defined so there is a lot of room to play with.
  3. Aftertouch : a shadow of the Keys type that adds pressure information. Could be redundant but it's available anyway. Same format as Keys.
  4. Bend : another shadow of the Keys for lateral pressure on the keys. A fun novelty. Same format as Keys.
    A second Bend might be possible, like BendY and BendZ...
  5. Mixing table : a collection of Knobs packets ?


Well, that's about it for this class, there is not much else to add, but there is room for extension.  Knobs can represent about anything anyway, so other types would be required if a different data format is required, for example events for drums ?


Other possible type : Tunnel (to encapsulate other types of traffic, such as MIDI, files, sound, network data...)


The Management class

That's where the kludges are.

For convenience, the numbers are decreasing so I'll use increasing negative numbers at this moment.

-1 : Tick

It's just a header that is passed along the chain to synchronise all the other clocks downstream. No data payload. Upon reception :

If no tick received within 1 second, send your own tick (8-16Hz ?)

-2 : Ping

This is another header with no payload, that is used to enumerate the devices and their IDs in a chain.

When a device receives this header, it must send its own Ping header before forwarding the one(s) it received. This is to prevent storms of packets, FIFO stuffing/overflow, and it also preserves the order of the chain, unlike other methods. The device must then ignore other Pings for a second or until it receives several packets that are not Pings.

-3 : Label

This packet allows getting and setting the "label" of a device (when possible). This packet can be emitted when the user changes the label on the device itself, so the DAW can update its display. But the DAW can also send this packet to inquire and/or change the label remotely.

When receiving a Label message, check if the ID matches, otherwise forward.

If the ID matches, check the set/get flag : if the flag is "set" then update the device's local label.

Then confirm by sending a Label message containing the device's label (1 size prefix byte and 0 to 255 UTF-8 bytes).

-4 : Serial

Get the read-only identification of the device : manufacturer, model, revision, serial number...

When receiving an empty Serial message with an ID that matches the device, the device sends a Serial message with a UTF-8 payload containing the detailed information.

More options or details could be obtained or selected using the Flags field. TBD.

-5 : Capabilities

This is where it gets somewhat MIDI-ish but capability enumeration is not strictly necessary because the available data streams are output anyway.

Maybe this could even be used to "tune"/enable/disable/configure each stream type from the device (sensitivity, max/min values, speed...)

-6 : Rate

Manage the update frequency, the relative priority for stream insertion...

-7 : JUMP

The timestamp counter is reset to a new value. Could also be used to extend the duration of a recording or stream.




This numbering with 2 ends (management at the Max of the range, and instruments at the bottom) allows the numbering and attributions to grow towards the middle of the range, at their own pace, while allowing efficient table-driven decoding of the most frequent messages.