close-circle
Close
0%
0%

LipSync

An assistive tech which allows quadriplegics to use touchscreen mobile devices using a mouth-operated joystick with sip and puff controls

Similar projects worth following
close
Touchscreen devices have become a staple piece of technology in this day and age. Most people cannot get through the day without using their smartphone. It keeps us connected. It keeps us organized. It is an integral part of our social lives. For people in wheelchairs whom experience difficulties with fine upper body motor control, the usage of mobile devices can be very challenging. Statistics show there are over 1 million people in the United States and Canada that have very limited or no use of their hands, making touchscreen devices very difficult or impossible to use.

The "LipSync" is an assistive technology device which is being developed to allow quadriplegics and other people with limited hand use the ability to use touchscreen mobile devices by manipulation of a mouth-operated joystick with integrated sip and puff controls. We are releasing all of our work open-source, to make the Lipsync a solution that can be made at the community level for less than $300.

OVERVIEW

In 2016, upwards of 1.6 billion people in the world are using smartphones. Smartphones are becoming a staple piece of technology for many citizens in North America and around the World. Smartphones and touchscreen devices enable users to navigate around their city better, to communicate with others more freely, and to operate applications which can promote happy and productive lives. Currently, there exist a limited number of practical devices for quadriplegics to use touchscreen devices - this is where we step in.

The LipSync is an electronic device which allows quadriplegics the ability to use compatible touchscreen and computer devices without the use of their hands. The user is able to manipulate a cursor on their device screen using a mouth-operated joystick with integrated sip and puff controls to simulate the actions of "tap" and hitting the back button, respectively. With longer sips and longer puffs, additional secondary features are enabled including a "tap and drag", "long tap and drag" and the possibility of more specialized functions as per the user's needs.

The LipSync is design specifically for portable devices, it does not require AC power, but it will work with any device including desktop and laptop computers that support mice through a universal serial bus (USB) or the Bluetooth connection.

The LipSync is an open-source hardware project where all of our 3D printer files, component lists and microcontroller code are made public. In the spirit of accessibility, our housing can be 3D printed, the electronic components are readily available and the assembly is as straightforward as possible.

The LipSync was envisioned as a holistic solution that takes in to consideration not only the interface but how the system is to be mounted on the user’s wheelchair. The actual electronics of the device is on part of the implementation. There are no standardized methods to which wheelchairs are designed. Wheelchair manufacturers can use round or square tubing. They often also use tubing which is not compatible with other manufacturers so that customers must buy accessories from them. As a result, there is not a standard location or clamping mechanism to mount assistive technology on the chair.

Wheelchairs are also customized to the user, including the height, width and seating position of the user. The seating on the wheelchairs are customized to minimize the incidents of pressure sores. As a result, the mounting system for assistive technology such as the LipSync also has to be customized. In addition to the instructions to assemble a LipSync, we have included instructions on how to mount a LipSync. A combination of off-theshelf and custom 3D printed components are provided in order to help makers create a fully integrated and customized solution for the user.

Now the maker community and disability community can meet, collaborate and work together constructing a LipSync over a period not much longer than a weekend. We hope these new relationships will continue spurring innovation within the maker community.

There are 3 main aspects we will be addressing in our project:

  1. Developing an easy to build, but robust electronics assembly that novice to experienced makers can build;
  2. Developing a device housing which can be 3D printed by makers either at home or at dedicated facilities;
  3. Creating mounting options for a variety of wheelchairs with 3D printed parts and/or commercially available components.

SMARTPHONES AND ACCESSIBILITY

Smartphones, by their very nature, are intended to be used while on the go. Traditional assistive technology designs for the desktop and laptop computer are not portable in nature, so they cannot be easily applied to smartphones. While single and dual input systems exist for smartphones, they are slow and frustrating to use for users with more capabilities of movement. Single and dual input switches are appropriate for users that can only make one or two movements consistently. The LipSync is...

Read more »

  • The Year in Review

    Chad12/21/2017 at 20:15 0 comments

    What a fantastic year it has been. We started off with the Access Makeathon, a 48-hour challenge where Makers worked together to create innovative assistive technologies for people with disabilities. And then we spent the entire year hopping around North America to meet dedicated Makers and build devices.

    10-year-old Timothy

    Above: Timothy, a 10-year-old with an anoxic brain injury, tests out an accessible Wii controller that a team built for him at the Access Makeathon

    We hosted or co-hosted 17 events, of which five were in the United States. With the help of many bright Makers, students, engineers, and tinkerers, we built 320 LipSyncs this year!

    Jim tests out a LipSync

    Above: Jim, who has a spinal cord injury, tests out a LipSync that students made for him at a Burnaby School District Makeathon

    Google and Neil Squire Society staff pose in front of the Android statue at Google HQ

    Above: Google and Neil Squire Society staff at Google headquarters for a buildathon during Accessibility Week

    Thank you to everyone for their skills, enthusiasm, and dedication towards making this happen. If you want to help in other ways, please consider making a donation this holiday season to support people with disabilities. Happy holidays and a wonderful new year!

  • The Youngest Person to Build a LipSync

    Chad12/15/2017 at 00:14 0 comments

    We were in Calgary last weekend for a buildathon with STEM Learning Lab. Around 25 West Island College students attended to help out, of whom only three had soldered before. But they were such brilliantly quick learners that by the end of the day, they had built 17 LipSyncs together.

    Students building LipSyncs

    Above: A group of students building LipSyncs

    Although the students worked in teams, an 11-year-old named Isabel built a device all by herself! She’s the youngest person yet to build a LipSync.

    Chad Leaman with Isabel

    Above: Chad Leaman, our Director of Innovation, with Isabel, the 11-year-old whiz

    Our speaker at the event was Mike, who has amyotrophic lateral sclerosis (ALS, also called Lou Gehrig’s disease). He has been inventing a range of devices to help him increase his independence.

    Mike brought with him his newest prototype, which is designed as a forklift for plates. It allows him to pick up a plate, put it into a microwave, take it out, and then bring it to his foot-controlled feeding device.

    Mike, Maya and the prototype

    Above: Mike with his daughter, Maya, alongside his prototype

    For more photos, please view our Facebook album.

  • We Fixed 67 LipSyncs

    Chad12/07/2017 at 22:20 0 comments

    On Monday, December 4th, we were at Vancouver Hack Space to repair a few LipSyncs. “A few” turned out to be 67: that’s how many LipSyncs are up and running now with the help of some fantastic Makers, all in 10 hours.

    The completed LipSyncs are on their way to people who need them. A big thank you to everyone who came out to troubleshoot with us!

    Makers working on LipSync parts

    Above: Makers putting their skills to good use. (Background, left) Luke, a Director at Vancouver Hack Space; (background, right) Marcello, a TELUS team member who also attended a previous buildathon; (foreground, left) Martin, an electrical engineer; (foreground, right) Charles, Mechanical Engineer at the Neil Squire Society

    Luke working on a LipSync

    Above: Luke working on LipSync parts

  • Repair a LipSync Night

    Chad11/30/2017 at 23:54 0 comments

    On Monday, December 4th, we'll be at Vancouver Hack Space to fix some LipSyncs. We've held a number of buildathons over the past year at schools, at corporate team-building events, and in makerspaces. Some of the LipSyncs built at these events need a bit more work before they are released into the wild.

    If you can solder, have some technical troubleshooting skills, and have some time, help us fix a LipSync or two!

    Makers working on LipSync parts

    Above: Makers working on LipSync parts at a previous build event

    RSVP on the Meetup page. Come any time, for as long as you are able.

    When: Monday, December 4th, 12pm - 10pm
    Where: Vancouver Hack Space, #104-1715 Cook Street Vancouver, BC V5Y 3J6

  • Sustainability TV Filming

    Chad11/23/2017 at 18:20 1 comment

    On Wednesday, November 22nd, Sustainability Television visited our Burnaby office to film a segment. The segment will focus on Neil Squire Society and Makers Making Change. It won’t be broadcast until September 2018, but we were excited to be a part of it!

    Here are some photos:

    Chad sitting in a room being prepped by a crew member

    Above: Chad, our Director of Innovation, being prepped by a crew member for his bit of the shoot

    Milad demonstrates a LipSync

    Above: Milad, our Systems Developer, demonstrates a LipSync as crew members film him

    Zee being filmed in front of a Makers making Change backdrop

    Above: Crew filming Zee, our Makers Making Change Project Manager

    We also want to talk about the Make Change Conference in Toronto on November 12th. Zee presented alongside Sorlie Madox, a talented artist and potter with multiple sclerosis, and Kat Singer, a multi-disciplinary community-based artist, activist, and educator.

    (left) Zee standing by a poster; (right) Zee at the panel

    Above: (left) Zee next to a Toronto Maker Map poster; (right) Zee at the panel with other speakers

    Together they discussed making for, with, and as people with disabilities. It was great to see how many passionate makers and organizers are involved in the Toronto maker community.

  • Makers Making Change Regional Coordinator Openings

    Chad11/20/2017 at 19:10 0 comments

    We are hiring! Our Makers Making Change initiative is looking for Regional Coordinators for our Prairie, Central, and Atlantic offices. The Regional Coordinators will work to integrate Makers Making Change into the culture of various makerspaces and hackspaces in their region.

    The Regional Coordinators will be a key part of our new Social Innovation department. They will implement and promote the program regionally by coordinating events, acquiring local publicity, and developing relationships as well as crucial partnerships locally with potential project partners.

    You’ll love this role if you are a maker who is actively involved in maker culture – someone who is passionate about creating in all aspects of life. You will have the opportunity to interact and work with diverse groups of people, including people with disabilities, engineers, designers, and occupational therapists.

    Click here to read more about the openings. The deadline is November 30th, 2017.

  • Google Builds 13 LipSyncs

    Chad11/20/2017 at 18:47 0 comments

    The LipSync project is funded by Google.org. Members of our Makers Making Change team headed down to Google HQ in Mountain View, California, for a Buildathon! Google was celebrating Accessibility Week and we joined them on Wednesday and Thursday, November 1st and 2nd.

    Eight organizations in the accessibility sphere, including the Neil Squire Society, were invited on Wednesday. On Thursday, Google staff built 13 LipSyncs for people with disabilities.

    Here are some photos from the event:

    Harry and Charles standing by an Accessibility Week poster

    Above: Neil Squire Society staff members Harry Lew, Manager of Research and Development (left) with Charles Gallagher, Mechanical Engineer

    We worked in an area called The Garage, a workshop space with a lot of equipment and tools.

    Google sign mounted on the wall

    Above: The Google sign in The Garage, made of old car parts

    Google staff working on LipSync parts

    Above: Susan Porter, an Occupational Therapist at Santa Clara Valley Medical Center, speaks as Google staff build LipSyncs. Among the staff members: (far right, in grey) Aaron Cunningham, Program Manager of The Garage; and (right, in black) Maya Ben-Ari, Product Manager of Android Accessibility.

    Team standing before Android statue

    Above: It’s always a good time for a group photo at the Android statue

    Thank you to everyone at Google for a fantastic Accessibility Week!

  • Customized, Open Source Assistive Technology Introduction

    Chad11/07/2017 at 00:39 0 comments

    On November 9, we’ll make a beeline for Downey in Southern California for the Customized, Open Source Assistive Technology Introduction. This free event is supported by Rancho Los Amigos National Rehabilitation Center, the Neil Squire Society and Makers Making Change.

    We’ll provide an overview of open source assistive technology and answer questions such as:

    • What are open source designs?
    • How do I find out what is available in the online libraries?
    • What is a 3D printer?
    • How do I find commercial 3D printing sources?
    • How do I modify these models to meet the end users’ needs?
    • What resources are available to get designs made if I do not have the skills or equipment to do so?
    • How do I find online networks of volunteers to make assistive technologies in my community?

    Sign up on the Eventbrite page here.

    When: Thursday, November 9, 2017; 11:00am – 1:00pm (light lunch provided)

    Where: Rancho Los Amigos National Rehabilitation Center, Support Services Annex (SSA), Room 1150, 7601 East Imperial Highway, Downey, California 90242, United States

    Rancho Los Amigos National Rehabilitation Center is one of the largest comprehensive medical rehabilitation centers in the U.S. providing services to a wide range of individuals with catastrophic illnesses and injuries. Rancho specializes in helping patients with disabilities regain skills and learn techniques to accomplish the basic activities of daily living, and returning to work or school if possible.

  • LipSync Buildathon with TELUS in Calgary

    Chad11/06/2017 at 20:10 0 comments

    We were in Calgary for our LipSync Buildathon with TELUS on Friday, October 27, followed by a trip to the Maker Faire Calgary over the weekend.

    At the Buildathon, we worked with TELUS staff to build 25 LipSyncs. The Honourable Kent Hehr, Minister of Sport and People with Disabilities, attended our event to speak about the role of makers of assistive technology in creating opportunities for people with disabilities.

    Group photo

    Above: Minister Hehr with Gary Birch; Neil Squire Society and Makers Making Change staff; TELUS team members Jees Joseph and Derek Keturakis; and volunteers

    Click here to view a Facebook video from Minister Hehr, talking to our Executive Director, Gary Birch, about assistive technology solutions.

    Some assistive tools on a table
    Above: Some pieces of assistive technology. Among the devices: (top, green) an implement that holds a pen and (centre right, red) a key holder.

    For more details, click here for a rundown of the Buildathon from the National Post.

  • Coming Up: TELUS LipSync Buildathon

    Chad10/26/2017 at 21:07 0 comments

    This Friday, October 27th, we’ll be in sunny Calgary for the TELUS LipSync Buildathon. Around 20 TELUS employees will work together to build 20 LipSyncs, supported by Makers Making Change. Teams will be supplied with the tools, components, parts and instructions.

    We will also have the pleasure of meeting The Honourable Kent Hehr, Minister of Sport and Persons with Disabilities, again, as he talks about the role of makers of assistive technology in creating opportunities for people with disabilities. He had previously met us as part of our travels for the TOM:Calgary Makeathon in August earlier this year.

    You are invited to join in this Buildathon! Click here to reserve a ticket.

    When: October 27th, 9:00am – 4:00pm
    Where: TELUS, 2912 Memorial Dr SE, Floor 1, Calgary, AB T2A 6R1

    Minister HehrAbove: The Honourable Kent Hehr, Minister of Sport and Persons with Disabilities, at the Tetra Society's Calgary location (photo courtesy of Minister Hehr)

    Here are some throwback photos from our TELUS Buildathon in June earlier this year, where employees participated to construct 30 LipSyncs as part of the TELUS Day of Giving.

    TELUS employees working on LipSync partsAbove: The Makers working on LipSync parts

    Below: The Makers with their completed LipSyncs
    The Makers with their completed LipSyncs

View all 43 project logs

Enjoy this project?

Share

Discussions

Mark Brady Ingle wrote 07/02/2017 at 22:52 point

is anyone still working on this project?  I need some help and ordering information on the mouth piece parts

  Are you sure? yes | no

Chad wrote 07/08/2017 at 16:14 point

hey Mark glad we caight up on email..sorry not logging into hackaday as often as I should. 

  Are you sure? yes | no

Mark Brady Ingle wrote 06/27/2017 at 02:50 point

I am testing on Mac and need the instructions for the RED/Green flash sequence.  I have a bluetooth connection established but I am not able to go any further.  Help would be appreci

  Are you sure? yes | no

Trent Robertson wrote 05/30/2017 at 16:00 point

Thank you for sharing this project. One of my best friends has ALS and would benefit from this. I have already set up voice recognition features in her apartment to allow her to control lights, air conditioning, and the TV, but she still needs someone to hold and control a tablet for her if she wants to surf the internet. I hope to find the time to build this soon.

  Are you sure? yes | no

Chad wrote 07/08/2017 at 16:13 point

hey Trent let us know if we can send you some parts / the main boards!

  Are you sure? yes | no

Mark Brady Ingle wrote 05/09/2017 at 20:53 point

Hello team!  Are the current instructions online the most current?  I have enough parts to get started so I wanted to make sure. 

  Are you sure? yes | no

Chad wrote 05/11/2017 at 20:20 point

Hi Mark, we have added the most recent as of now.  Let us know if you got questions with the build and in particular ping @Ivan.Gee if you need in particular details

  Are you sure? yes | no

eroy wrote 01/02/2017 at 23:34 point

Thank you for this project! I am endlessly disappointed in the cost of assistive technology. To make something like this open source is a gesture rarely seen. 

  Are you sure? yes | no

Charles G wrote 02/26/2017 at 06:49 point

Thank you for the kind words eroy! Stay tuned in the next couple weeks as we launch a new and improved version of the LipSync at an even lower price point!

  Are you sure? yes | no

Manuel Z wrote 10/10/2016 at 19:18 point

this is an amazing project! congratulations!!!! 

  Are you sure? yes | no

Charles G wrote 10/10/2016 at 23:14 point

Thanks, Manuel!

  Are you sure? yes | no

Sadiq Mohamed wrote 09/22/2016 at 12:02 point

A very worthwhile project. I had a friend with MS who would have benefited from this. Unfortunately he died some years ago. I wish you the best of luck with your testing and look forward to the results.

  Are you sure? yes | no

Ivan wrote 09/22/2016 at 17:55 point

My condolences about your friend @Sadiq Mohamed . I am glad you believe in our project, we are working our tails off over here so we can start getting makers and users together to start building these and help increase people's quality on life. Just as a side note, our plans are to continue compiling and refining open-source assistive technologies, if you have any ideas I would love to hear them.

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates