Close
0%
0%

HandShake

Enabling communication through gesture

Similar projects worth following
The goal of this project is to enable people with mobility disability to use gesture to control switchable software. Software like Sensory Software's Grid 2 enables communication using a single switch. Some people are too disabled to use regular controls, such as buttons, to operate this software. This project aims to use whatever hand gesture that can comfortably be made as a controller for this software.

The need for this was identified by the Technologists and Occupational Therapists at Beaumont College, Lancaster, UK. Beaumont College educates about 100 students with a range of disability. We now have a working prototype being tested at the college. This uses the BBC microbit boards One board is worn on the student's wrist. When the student shakes their hand, the microbit on the wrist sends a radio message to a second microbit, which is attached to the communication device.

The sensitivity of the gesture to be detected can be tuned to the motion that each individual can comfortably make. Development is taking place between InfoLab21, Lancaster University and Beaumont College, Lancaster, UK. The students at Beaumont have a variety of physical mobility issues, mostly as a result of cerebral palsy. This means that I can't ask them to replicate a predefined gesture. I need to be able to recognise whatever gesture each of the students can make. The aim is to use HandShake as a virtual switch to enable the student to interact with their communication devices. The college uses Sensory Software's Grid 2 communication software. This type of software can be operated with a single switch to create speech or operate environmental controls. So giving somebody a single switch can make a difference to their ability to communicate.

Preliminary testing can be seen here:

https://youtu.be/gRrBVgBbUFQ

The need for this project was identified by the Technologists and Occupational Therapists at Beaumont College.

The latest manual with build details and code can be found on my github at:

https://github.com/hardwaremonkey/microbit

Some of my other projects and earlier work on this project can be found at https://sites.google.com/site/hardwaremonkey/  

With thanks to InfoLab21, Lancaster University and Beaumont College, Lancaster for all of the help with this and my other projects.

handshake_manual.odt

Check the github repository for the latest version of the manual and code.

text - 3.33 MB - 08/28/2017 at 18:47

Download

  • 2 × BBC micro:bit One will be the transmitter, the second will be the receiver
  • 1 × BBC micro:bit AAA battery holder with batteries
  • 1 × USB micro cable to connect the receiver microbit to your laptop
  • 1 × BBC micro:bit wrist holder, home made build instructions at: https://sites.google.com/site/hardwaremonkey/blog/microbitwristholder
  • 1 × Smart phone arm band $5 from eBay, to make the wrist holder

View all 9 components

  • Build Instructions and Ramble

    matt oppenheim08/24/2017 at 18:47 0 comments

    Build Instructions on github

    I decided to put the build instructions and other documentation into an OpenOffice document. Why not put it all up on the Hackaday project instructions? Having the OpenOffice document allows me to put it up on my github site. Anybody else can then download it and modify it or take bits from it to help with their project or perhaps in modifying this project. I'll put a version of this onto the Hackaday project site, but you're best off going to the github site and downloading the latest version.

    There was a bug on the Hackaday site which made it difficult to put up my build instructions. I found that I couldn't edit the instructions after the initial post. I had this bug verified by Hackaday. It is probably fixed by now.

    Ramble

    Is this project a hack? The build complexity is low compared with many other projects. The Hackaday site defines hacking as:

    Hacking is an art form that uses something in a way in which it was not originally intended. This highly creative activity can be highly technical, simply clever, or both.

    I am re-purposing an educational tool to be assistive technology, so I think it qualifies as a hack. What the reader doesn't see is my many, many failed approaches.  I spent some months writing code for the Microsoft Kinect, then the Leap Motion. Neither were suitable for my end user group. Then I made several iterations of proper kludges, lashing together Micropython boards, IMUs and XBees. These would undoubtedly qualify as 'hacks'. My final implementation (so far...) has the advantage of being simple. I've put the time and effort in that many more sophisticated projects on this site have, though the end product does not appear to reflect this as it is based on unmodified off the shelf boards. 

    Getting from A to B, I visited most of the other letters in the alphabet.

  • licence

    matt oppenheim08/05/2017 at 18:13 0 comments

    One of the requirements for the 2017 competition entry is to detail licence requirements. I don't know much about the different types. When I set up the github account for this project, I was asked to choose a licence. I went with the MIT licence as it gives free use of the code so long as I am quoted as the original creator. Why would this appeal? Do I crave the fame of having this project associated with me? Of course! In reality, knowing who created the code will allow anybody who uses it to easily find my webpage, where I intend to keep the designs and code up to date with the inevitable fixes and improvements. This should help whoever uses handShake to have easy access to the latest and most reliable design and code. 

    Here we go:

    Copyright (c) 2017 Matthew Oppenheim

    Permission is hereby granted, free of charge, to any person obtaining a copy
    of this software and associated documentation files (the "Software"), to deal
    in the Software without restriction, including without limitation the rights
    to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
    copies of the Software, and to permit persons to whom the Software is
    furnished to do so, subject to the following conditions:

    The above copyright notice and this permission notice shall be included in all
    copies or substantial portions of the Software.

    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
    AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
    OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
    SOFTWARE.

  • HandShake, a history

    matt oppenheim07/28/2017 at 17:31 0 comments

    I set up a hackaday project page in '16 for the competition that year, but didn't make enough progress to get anywhere. But I'm back! This year I started to use the BBC Micro:bit. The microbit is a single board aimed at 11-12 year old school students. I think of it as a gateway drug to embedded hardware. It is designed to be a stepping stone to give somebody who has no interest in computers or programming the chance to realise how empty their life is without these interests. I imagine the developers have a different description.


    The microbit has an embedded microcontroller and a few sensors including an accelerometer. The 'killer feature' is the board to board radio. It 'just works'(TM). As the boards have been tested and deemed safe to use at schools, this means that I can use them without worrying about the reprisals that follow after inflicting my own kludges onto unsuspecting victims.

    I did prove the HandShake idea with a custom kludge of an xbee, accelerometer, battery and regulator. If you go down my project page, you can read of my many, many failures. How I failed. I failed Mightily. I'd like to say I emerged a better person for the struggle, but I suspect it all contributed to the bitter, introverted, golem like creature I am today.

  • HandShake testing at Beaumont

    matt oppenheim07/27/2017 at 17:55 0 comments

    Jordan and Jack, who are students at Beaumont, kindly volunteered to have a go at testing HandShake. The aim was to turn on an LED light through hand gesture. Please have a look at the video below. When the students move their hand, the gesture is detected and the blink1 USB LED flashes. The real aim of HandShake is to enable the students to operate their communications software. In the video you can see that the two students can move their hands with different strengths. HandShake can be tuned to the sensitivity level needed for each student using the buttons on the front of either of the boards.

View all 4 project logs

  • 1
    Build instructions are in the manual on the github site.

    Build instructions and a user guide are in the manual on the github site. I put a draft of this manual on to this site as well, but you are best off getting the most up to date version from github. This manual is an OpenOffice document.

View all instructions

Enjoy this project?

Share

Discussions

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates