Vocal Trainer

The "Vocal Trainer" purpose is to facilitate sport practice to visual impaired people.

Similar projects worth following
Today, in fitness clubs, training devices are lacking of accessibility: screens are too small, not enough light or contrast, keys are too small and not readable, flat keyboards,...
To overcome these issues, the idea is to command these devices (like a treadmill) using the voice, through an earplug or similar device.

The project aims to replace completely the console with voice enabled interactions.
The "Vocal Trainer" can give basic information like distance or speed, as a response to "get distance" or "get speed".
The user can command the treadmill with : "speed up", "speed down", "stop",...

The "Vocal Trainer" is hacking into the keyboard of the training device; it makes the appropriate contacts to replace physical key presses. The keyboard remains usable for any normal person.
By using this approach, it can be installed on any training device.

The project started with the research of an autonomous voice recognition library, that would take care of the audio input stream, and run smoothly on our embedded platform.

Of course we could have used an online service for that (Google,...) but there were 2 main drawbacks:
- the response delay that can vary, and be too long
- the risk that the internet connection shut down
These are unacceptable issues while running on a treadmill ; you don't want to loose control of it...

The ideal library would match all of the following criteria:
- lightweight: the CPU should still have some room for treadmill control
- have a python interface: this is our preferred language
- be open source
- be free charge: we do not have much money :-)

Very soon it turned out that CMU sphinx, and its small sister "pocketsphinx" would fill the bill.

As embedded platform, we just use an Odroid-C1, and made a nice 3D printed case for it.
This platform has plenty of I/O pins, USB ports, ethernet,...
It is just missing bluetooth (note for next platform selection), but we can easily plug in a bluetooth dongle on it. So, we can connect a wireless earplug, and complete our input channel.

This box will be mounted on the back of the treadmill console.

This was the easy part. Next, the question is: how to interface with the treadmill ?

Follow us on Facebook, and via our web site.

  • 1 × Odroid-C1 embedded platform
  • 1 × Plantronic earplug noise cancelling quality earplug

  • MakerFaire Rome 2016

    laurenternes10/09/2016 at 13:29 0 comments

    Meet us at the MaferFaire Rome 2016.

    Location: B10 (pav. 6)

    And try the treadmill yourself !

  • Some tuning...

    laurenternes10/02/2016 at 19:54 0 comments

    Voice control is tricky...

    Even with the simple words "start" and "stop", the system confuses too much and understand "stop" when we say "start" (hopefully not the inverse !)
    This is mostly due to noise sensitivity, and the fact that I do have a poor English accent :-(

    We had to change "start" for "engage" which is much efficient and stylish.

    We go on with fine tuning, and try to find the best dictionary of commands, with the less false positives.

    Then, go and explore worlds, where no blind has gone before.

  • Remote control

    laurenternes10/02/2016 at 19:38 0 comments

    The interface board is ready !

    For the record, we use a small CNC to mill and drill the PCB. This is actually also a 3D printer: the Fabtotum. This is a very cool little machine that I back during their Kickstarter campaign.

    It is now mounted with hot glue (ah ! hacking !) inside the console. The optos are connected via a flat cable to the keyboard solder joints (on the bottom right).

    Ready to go !
    We can now close the console, and use this remote control the Odroid.

    The Odroid is mounted (with screws this time) on the back of the console, we added a bluetooth dongle (for the earplug) and a USB cable. (so far, the console was shown upside down)

    The vocal stack is ready, prototype mode. Some simple words: "start", "stop" that maps to the same keys on the console. Everything running on a debian.

    The Odroid is controlled by a laptop through a SSH connection, this is for easier logging and debugging.

    Now, time for live testing, fine tuning,...

  • Keyboard decoding

    laurenternes10/02/2016 at 18:45 0 comments

    Well, this is the tricky part.

    I once found a very interesting cabling problem on the internet.
    It went like that:
    - you have a long cable making the connection across a sea.
    - this cable contains a lot of wires and for some reason the manufacturer forgot to label them. So, there is no way for you to know which wire is which from one side to the other of the sea.
    - the problem is to identify them with as little steps as possible. You can use a multimeter, and you can travel overseas to make some connections (with local wires only). But you have to limit the number of travel to the bare minimum.

    This is exactly the kind of thing I had to do... Moreover, I had no clue about the way the keyboard was organised (it could be a matrix, multiple contact keys, or a mix of them).

    Hopefully I knew the answer to the initial problem. Given a flat ribbon, a multimeter, key-presses, and a big bucket of patience I was able to sort it out.

    Each key of the keyboard makes actually 2 connections with a ground plane. If one wants to simulate a key-press, just connect simultaneously these 2 wires to the ground, and voilà ! Also, further experimentation showed that timing is important; the contact must be no less than 100ms.

    Now that we know how to spoof the keyboard, we have to choose how to do it:
    - we can use directly the Odroid GPIO
    - we can use some interface board

    If you know what you're doing when hacking some equipment that is connected to the mains, or simply went through miserable failures where your contraption mysteriously catches fire, you should know about current loops and isolation.

    The minimum we could do is to use optocouplers to fake key contacts. So we went for an additional interface board. This board controls optos on one side, and receives commands from a USB on the other side, using a software USB stack.

    The board is placed inside the console where there is still some spare room.

    After a couple of iterations and careful measurements, we have our design.

    With this board inside the treadmill console, we achieve exactly what we were missing: an external connector for remote control. The advantage of using a USB connection (instead of I2C, or other) is that it can be controlled by a PC as well. So, it eases further development and testing of the control software; we are not forced to develop directly on the Odroid.

  • HW interface

    laurenternes10/02/2016 at 14:21 0 comments

    As a hacker, the problem with electronic equipments is that you usually don't have the connections you want them to have to be able to play with it.

    So, is it for a (low-cost) treadmill. Unless you target a medical grade equipment, you won't find any external connector for remote control. As far as I know, if you find something, it will be a proprietary connection with undocumented protocol.

    As we wanted to make a proof of concept (and we were low on budget) we didn't want to play with expensive material, and also we wanted to be sure to reach some result.

    So we bet on a very pragmatic approach: keyboard hacking. No tricky serial connection, or encrypted data, just plain old key-press spoofing.

    Or remark thought: we could have chosen to leave the console untouched, and use a bunch of motors to mechanically press the keys. Sure it would have been elegant as well, maybe a bit more bulky, and we a no personal fan of mechanic. Also, we couldn't resist to know what's inside this cheap device, and inject some magic in it.

    Obviously, the sole act of opening the treadmill console has voided the guarantee, but at least we became the master of the beast.
    No surprises in the inside, for a low cost BOM: a couple of chip on board, multi-product PCB with unpopulated components, Chinese labels...

    On the bottom-right of the PCB we can see the flex cable coming from the keyboard.
    (For info, the top wires are for safety key switch, and the other right connector is for power and motor control)

    Next, the decoding of the keyboard.

View all 5 project logs

Enjoy this project?



Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates