Close
0%
0%

Litar: LiDAR Air Guitar

Compact, handheld air guitar with four LiDAR strings and Bluetooth

Similar projects worth following
Man has always enjoyed jamming to their favourite riffs and solos with their imaginary air guitar ever since the age of fire. The unfortunate downside to this form of music creation is that while it releases endorphins, you can't actually make any useful sounds from it.

Introducing the Litar, a you beaut wireless air guitar. Four individual LiDAR air strings capture your strums which allow different neck positions and strings to be played. Coupled with a low latency input and Bluetooth MIDI interface, you can jam away or play with the finest control on your phone or laptop.

The Litar has a simplified, but versatile pressure sensitive chord interface, that allows for advanced control and aftertouch expression. Coupled with MIDI communication as well as an Arudino development environment, the musical possibilities are many.

Jam on air for real!

Rockin' on with the Litar.
Original photograph by Frank Schwichtenberg. Remixed
under CreativeCommons*.

Building on from last year's Hackaday prize entry the MappyDot, the Litar makes use of the VL53L1x LiDAR sensors from ST Micro to create an array of air strings that you can interact with.

The sensors are aligned in such a way that three sensors see a single string in space; two sensors define the string border and a third sensor obtains the string distance. This enures that you don't get miss plucks and it also allows the strings to be packed tighter together over one or two sensor setups (individual sensors have a 25 degree field of view). To visualise this, the image below shows the sensor field of views, with the grey areas highlighting the virtual air strings:

With each string, the Litar also obtains the distance that the string was plucked at, which can be directly fed into a MIDI channel for processing.

The Litar isn't just about a sensing array though. Smart code algorithms need to process the incoming values in the lowest latency possible to before you're smooth sailing. The human brain generally works well with latency values under 50ms, although anything over 20ms is considered perceivable by some. The sensors themselves measure at 100Hz (10ms), so there's not much room to breath. Throw Bluetooth into the mix and you're on the highway to hell'a bad latency if you're not careful.

Coupled with this requirement, is the design of an ergonomic grip and chord input. Various prototypes have been tested already, but it's a long way to the top if you want to design an easy to use and strain free interface that fits most.

Once everything is working, hardware kits will be made available so you can get your hand on it and strum away.

All Litar code on these project pages or subsequent code repositories are licensed under the GNU General Public License (GPL) unless otherwise stated. Documentation on these project pages are licenced under the Creative Commons Attribution-Share Alike 4.0 license.

* Modification to image includes cropping and Litar in hand.

  • Not so Soothing Tunes but the Hardware Is Sound

    Blecky10/07/2018 at 10:07 0 comments

    Here's a quick video of the Litar in action. You can see the Litar is working with battery power over Bluetooth LE MIDI to FL Studio (through a loopback driver since FL Studio doesn't support Bluetooth MIDI yet).

    There's still a fair bit of work in the coming weeks to getting a sensible chord and scale mapping for the strings and buttons, as well as velocity and aftertouch effects to actually play some decent music, but the main hardware components and string detection algorithms are working as intended.

  • Decreasing Sensor Parallax

    Blecky09/03/2018 at 11:47 0 comments

    In the quest to reduce the latency of string measurements, it was discovered that there is significant sensor parallax due to each sensor measuring independently of one and other (clock drift over time, different start times). Notice the out of phase measurements below from two different sensors:

    This means that it may take up to 20ms (at 100Hz) to be able to operate on the latest sensor measurements from all sensors when detecting strings.

    To counter this, the MappyDot firmware required the addition of a synchronisation pulse to keep measurements in line with each other. The pulse is sent along the addressing lines from one device to the next (as soon as the pulse is received it sends it to the next device in the interrupt routine to reduce propagation delay).

    The master device controls the firing of each measurement when in synchronisation mode. So if it is operating continuously, or in single shot mode, each subsequent device in the chain will also fire at the same time. It has the added benefit of only requiring a single "take measurement" command to be sent to the master, when in single shot mode.

    So with this in place we get the following synchronised measurements without the parallax (measurement pulses shown from two random sensors):

    Yay, 10ms string measurements!

  • Boards Populated

    Blecky06/23/2018 at 09:50 0 comments

    The Litar boards are populated and ready for testing:

    Looking good!

  • Boards!

    Blecky06/13/2018 at 11:20 0 comments

    We finally have some boards for testing! The previous Litar prototypes were just a jumble of wires and protoboards.

    These boards contain a LiPo charger circuit, BLE module, hall effect sensors and USB connectivity. They connect up to the string of MappyDot Pluses with a small cable.

    It is hoped to release these as a kit sometime :D

  • Tasty Strings

    Blecky04/19/2018 at 10:10 0 comments

    Here's a bit of a taster of the strings working on the Litar. The LEDs light up when the corresponding string is being touched.


    There's 4 strings so it's the 4 middle sensors that light up (they are manually turned on with the LED on and off commands of the MappyDot).

  • 100Hz Sensor Measurements!

    Blecky03/28/2018 at 12:53 0 comments

    So it turns out, contrary to the information on the ST Microelectronics site, the VL53L1x sensor actually does run up to 100Hz at shorter ranges (<1m) pretty reliably after some testing (it was mentioned this was possible in some of their older press releases).

    Which means a halving of the sensor latency to 10ms. This is absolutely awesome!

  • MappyDot Plus Testing

    Blecky03/21/2018 at 10:58 2 comments

    As part of the new button case design, the Litar will also be getting an upgrade to the MappyDot Plus we have been testing, which uses the brand new VL53L1x sensor:

    The new sensor offers better ambient light immunity and an adjustable field of view. This will hopefully allow us to change the size of the strings programmatically and offer better string strumming recognition in noisy environments.

    The MappyDot Plus will be available on Tindie shortly - https://www.tindie.com/products/blecky/mappydot-a-smarter-micro-lidar-sensor/

  • Designing Ergonomic Buttons

    Blecky03/20/2018 at 06:51 0 comments

    The first Litar prototype used some 2D hall effect joysticks (with button click) as a demonstration to enable advanced chord control, vibrato as well as ambidextrous control:

    These joysticks were based on the AMS Easypoint N40P107 assembly which uses the AS5013 two-dimensional magnetic position sensor. These are great little compact assemblies, however unfortunately the manufacturer has decided to no longer make these and leave it up to the product designers to make their own magnetic pointer with the position sensor.

    This left one of two options, either make your own pointer assembly, or move to a low profile analogue joystick instead. Low profile analogue joysticks with a button click are unfortunately difficult to find it turns out, unless you want large quantities of them (like the ones in the 8bitdo pro controllers):

    I set that problem aside for another day and continued to play further with the Litar...

    After a while and letting different people play with it, it was found that lateral motion in your fingers on something you are holding onto with the same hand was not only difficult, but actually caused a fair bit of strain after a short amount of use. I tried moving the buttons around in the case to make them more ergonomic, but they were still difficult to play and removed the ambidexterity of the controller.

    So I set out to design something different. I wanted to be able to still do vibrato, so I needed a linear sensor of some sort, preferably with some sort of pressure control. Why not a linear hall effect sensor with springs and a simple rectangular button?!

    This sort of setup does simplify the chord input a fair bit, but it does still give you varying levels of control input and can still be waggled for vibrato.

    Here's a cutaway of what I came up with:

    The linear hall effect sensor sits on a PCB, as does two springs on each side of the button. The button moves freely in the hole of the case and is held there by the springs. There's two stoppers on the button part that hits the PCB as it comes down and these stoppers could press a momentary switch. There's a magnet in the center of the button as well.

  • How do the Strings Work?

    Blecky03/18/2018 at 13:00 0 comments

    The sensor array on the Litar has been designed to allow for the lowest latency and best accuracy possible when obtaining a string interaction. Each string has a specific geometry that allows them to be easy to play and makes them large enough to pluck.

    The sensors are aligned in such a way that three sensors see a single string in space; two sensors define the string border and a third sensor obtains the string distance. This enures that you don't get miss plucks and it also allows the strings to be packed tighter together over one or two sensor setups (the sensors have a 25 degree field of view). To visualise this, the image below shows the sensor field of views, with the grey areas highlighting the virtual strings:

    To obtain the strings, the data from the sensors is brought into an array of sensor distances over time and placed through a rapid-change detection algorithm. The algorithm scans each combination of sensors which create a string and then obtains the correct distance for the string being plucked. It does this by looking at the most recent value as well as its history to determine which string has been plucked, with the latency of a single distance measurement. With this you can also obtain entrance to the string area velocity information, without increasing latency.

    And that's how the Litar gets its strings.

View all 9 project logs

Enjoy this project?

Share

Discussions

Est wrote 03/22/2018 at 08:36 point

Your Idea is great! clever how you arranged the distance sensors. Time ago I was working on a project that i have abandoned with a IR distance sensor and a dsPIC that plays a sound.

I am looking forward to see a video of it in action as I am curious how it sounds.

great project!

  Are you sure? yes | no

Charlie Lindahl wrote 03/21/2018 at 17:56 point

Do you have a video of someone playing this instrument somewhere? Searching YouTube was an exercise in futility (there is SO MUCH STUFF there!).

  Are you sure? yes | no

Blecky wrote 03/22/2018 at 03:06 point

I should have a video this weekend for it working with the new Vl53L1x sensors :)


I was having issues with the previous prototype when I added the hall effect joysticks, as these were I2C based and were sharing the same bus as the sensors on an I2C hub (the joysticks had the same address) which added a bit of latency.

The new hall effect sensors use an onboard ADC instead and the I2C is used exclusively for the sensors, so it's much nicer.

  Are you sure? yes | no

Gray wrote 03/21/2018 at 17:07 point

Ohhhh this is so awesome!!! Well done!

  Are you sure? yes | no

Blecky wrote 03/22/2018 at 03:35 point

Cheers Gray, it's been a pretty fun project so far. There's a couple of things to iron out, but the technical side of things is going pretty well.

  Are you sure? yes | no

jean.perardel wrote 03/14/2018 at 10:21 point

haha, great idea ! 

it looks amazing :) 

  Are you sure? yes | no

Blecky wrote 03/14/2018 at 10:31 point

I saw the Musical Instrument challenge and definitely had to enter. It's still a prototype, but everything is working well. Unfortunately I'm not a musician, so I've focused on the hardware side with things like getting the latency down. But the strings strum and the Bluetooth, tooths.

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates