communication through gesture

A trigger is sent to a communications device when a hand moves above an adjustable threshold of acceleration

Similar projects worth following
HandShake is a project to help enable people who need to access software to create speech to be able to access this technology. Some people are unable to use physical controllers such as buttons and joysticks but are still able to make intentional hand movements. HandShake uses a pair of BBC micro:bits to detect this hand movement and send a trigger to switchable communication software to enable speech to be composed. A trigger is sent to the communications device when the user moves a hand above an adjustable threshold of acceleration.

Please see a video of hand shake being tested at Beaumont College, Lancaster, UK here:

hand shake testing video

Project details can also be found on my website at:

hand shake project site

The latest hand shake system uses the BBC micro:bit. One microbit is worn on the wrist and a second is connected to a laptop or communications device. When a gesture is made, the motion is detected by the microbit on the wrist. This microbit uses its radio to signal the microbit connected to the laptop. This microbit then triggers an event on the laptop through the USB cable connected to the laptop. The trigger can also be used as switch to control communications software, such as Smartbox’s Grid software. Thanks to Sensory Software for giving me a licence for this software to allow me to test the hardware with it.

The licence for this project is GNU GPL 3, link here:

GNU GPL licence v3

  • 2 × BBC micro:bit board About £12 each
  • 1 × wrist holder for one microbit board Make it yourself or buy one from e.g. Polulu
  • 1 × Freetronics Leostick (not required for most applications) This board appears as a physical keyboard to the PC, so can send 'real' keystrokes.

  • Running in Administrator mode

    matt oppenheim3 days ago 0 comments

    Sensory Software's Grid 3 is a popular communication software package, running in Windows. Naturally, I would like handShake to be able to operate this software through the software keystrokes that handShake generates.  To get Grid 3 to respond to a software keystroke, I have to 'elevate' handShake to run as an Administrator. I do have a second solution. I can use a Freetronics Leostick USB dongle as a pretend keyboard and have this generate keystrokes that appear as coming from a physical keyboard. I did this for a while, but this adds a layer of complexity and expense to the project. The simplest solution is to run handShake as an Administrator when using Grid 3, or other software that requires software keystrokes to come from an elevated source.

    I tested out adding the functionality for handShake to detect when Grid3 was running, then automatically try to elevate to run in Administrator mode. I got this running. Then removed the functionality. Why? Security. Software running as Administrator can damage your system if incorrectly or maliciously written.

    Now handShake detects if Grid 3 is running and advises that this requires the software to be restarted as an Administrator, but does not automatically restart in Administrator mode.

    Running handShake as Administrator is a choice that the user makes and implements if he or she deems necessary.  As all of the code is on the project GitHub site, the code can be reviewed by the user or whoever administers the communication device that handShake runs on. The code can be reviewed to check that it is safe to run as an Administrator.

    One of the advantages of open source projects is that they are open to scrutiny to find security flaws.

  • Testing

    matt oppenheim07/01/2020 at 17:07 0 comments

    I spent a short time working in mechanical engineering, for a company that makes quick release anchors for oil rigs. Every part was tested, as you can imagine. But not only the finished assembly. Even fragments of the castings used to construct the main component were taken for destructive testing to prove that the metal had the correct characteristics.

    Why should it be any different in software engineering? Software testing is an established part of producing reliable code. I have been slack at implementing this. So I took an online course through Pluralsight, kindly made available for free during the recent lockdown, on writing tests for software. The course is aimed at Python, which I use for scripting the software implemented on the PC.

    So I am slowly getting to grips with writing tests. I find writing stand alone doctests a useful tool. I test the code from the command line, then copy and paste the tests into a separate file. It is not quite as simple as this, but you get the idea. I am slowly getting to grips with mocks and pytest. Using the doctests is fast and easier, so I am more likely to actually write some tests.

  • What do I want out of posting this project

    matt oppenheim06/30/2020 at 15:18 0 comments

    There are so many excellent projects submitted to the Hackaday competitions, showing a level of skill that I don't have - why would I enter this? I hope that somebody who can use the project shown here, or who knows somebody that could potentially benefit from the project, reads this site and tries to implement hand shake. Please get in touch if there is anything that I did not explain well.

    If you do try this out, please let me know. I keep an unpaid position at my local university for when I am not working at sea. This gives me the resources and support to do this work. Any feedback helps me write publications, which I need to keep the position. The data will be anonymised.

  • Packaging python executables

    matt oppenheim06/30/2020 at 15:10 0 comments

    There are several libraries for making a python script executable. Some work for some folk some of the time. I am not sure if they all work for all folk all of the time. I used the cx_freeze library successfully with one of my projects. It looks to still be active. Currently I use pyinstaller.

    One issue I found using these packaging managers is that the size of the final executable can seem unreasonably large. Of course, a Python environment needs to be wrapped up in the executable. To minimise the size of this,  work in a virtual environment with only the extra modules installed that you need for your script to run. I got this tip from the website here in the post dated

  • Sensory Software Grid 3 control problems

    matt oppenheim06/29/2020 at 17:08 0 comments

    The Python script I wrote to interact with software sends a software keyboard keypress signal. Most packages respond to this. I am having trouble with Sensory Software's Grid 3 package. This softwrae only responds to a software keystroke when the window running the software does not have focus!  So if I click on the Grid 3 window, it will not register the software keystrokes transmitted by my script. If I click outside of the window, so that the Grid 3 software no longer has the 'focus', then it does respond to the keystrokes.

    Grid 2 would not respond to software keystrokes at all. I had some correspondence with the software developer over this a few years ago. He sent me some sample c# code to interact with the package, but I never got this to work. Which is totally a comment on my lack of coding ability and experience.

    That was the reason I set up the Leostick to spoof a physical keyboard key press. The Leostick is a miniature Leonardo board. It shares the ability to appear like a physical keyboard to the Windows operating system.

    So, I could set up my script to recognise when the Grid 3 software is in use and focus on e.g. a menu bar or something where the software keystrokes will not activate anything. This is all a bit kludgey. Or I could resurrect using the Leostick. With a Leostick plugged in, there will always be a response from whatever you are trying to control as the Leostick is a real keyboard as far as Windows is concerned.

    I listen to the Gnu World Order podcast. The presenter, Klaatu, mentions in one episode that your open source software can only ever be as robust as the weakest closed source element it relies on.

View all 5 project logs

View all instructions

Enjoy this project?



Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates