An assistive tech which allows quadriplegics to use touchscreen mobile devices using a mouth-operated joystick with sip and puff controls

Similar projects worth following
Touchscreen devices have become a staple piece of technology in this day and age. Most people cannot get through the day without using their smartphone. It keeps us connected. It keeps us organized. It is an integral part of our social lives. For people in wheelchairs whom experience difficulties with fine upper body motor control, the usage of mobile devices can be very challenging. Statistics show there are over 1 million people in the United States and Canada that have very limited or no use of their hands, making touchscreen devices very difficult or impossible to use.

The "LipSync" is an assistive technology device which is being developed to allow quadriplegics and other people with limited hand use the ability to use touchscreen mobile devices by manipulation of a mouth-operated joystick with integrated sip and puff controls. We are releasing all of our work open-source, to make the Lipsync a solution that can be made at the community level for less than $300.


In 2016, upwards of 1.6 billion people in the world are using smartphones. Smartphones are becoming a staple piece of technology for many citizens in North America and around the World. Smartphones and touchscreen devices enable users to navigate around their city better, to communicate with others more freely, and to operate applications which can promote happy and productive lives. Currently, there exist a limited number of practical devices for quadriplegics to use touchscreen devices - this is where we step in.

The LipSync is an electronic device which allows quadriplegics the ability to use compatible touchscreen and computer devices without the use of their hands. The user is able to manipulate a cursor on their device screen using a mouth-operated joystick with integrated sip and puff controls to simulate the actions of "tap" and hitting the back button, respectively. With longer sips and longer puffs, additional secondary features are enabled including a "tap and drag", "long tap and drag" and the possibility of more specialized functions as per the user's needs.

The LipSync is design specifically for portable devices, it does not require AC power, but it will work with any device including desktop and laptop computers that support mice through a universal serial bus (USB) or the Bluetooth connection.

The LipSync is an open-source hardware project where all of our 3D printer files, component lists and microcontroller code are made public. In the spirit of accessibility, our housing can be 3D printed, the electronic components are readily available and the assembly is as straightforward as possible.

The LipSync was envisioned as a holistic solution that takes in to consideration not only the interface but how the system is to be mounted on the user’s wheelchair. The actual electronics of the device is on part of the implementation. There are no standardized methods to which wheelchairs are designed. Wheelchair manufacturers can use round or square tubing. They often also use tubing which is not compatible with other manufacturers so that customers must buy accessories from them. As a result, there is not a standard location or clamping mechanism to mount assistive technology on the chair.

Wheelchairs are also customized to the user, including the height, width and seating position of the user. The seating on the wheelchairs are customized to minimize the incidents of pressure sores. As a result, the mounting system for assistive technology such as the LipSync also has to be customized. In addition to the instructions to assemble a LipSync, we have included instructions on how to mount a LipSync. A combination of off-theshelf and custom 3D printed components are provided in order to help makers create a fully integrated and customized solution for the user.

Now the maker community and disability community can meet, collaborate and work together constructing a LipSync over a period not much longer than a weekend. We hope these new relationships will continue spurring innovation within the maker community.

There are 3 main aspects we will be addressing in our project:

  1. Developing an easy to build, but robust electronics assembly that novice to experienced makers can build;
  2. Developing a device housing which can be 3D printed by makers either at home or at dedicated facilities;
  3. Creating mounting options for a variety of wheelchairs with 3D printed parts and/or commercially available components.


Smartphones, by their very nature, are intended to be used while on the go. Traditional assistive technology designs for the desktop and laptop computer are not portable in nature, so they cannot be easily applied to smartphones. While single and dual input systems exist for smartphones, they are slow and frustrating to use for users with more capabilities of movement. Single and dual input switches are appropriate for users that can only make one or two movements consistently. The LipSync is...

Read more »

  • Our World Create Day Plans

    Chad03/08/2018 at 22:17 0 comments

    March 17th is World Create Day: a day when hackers and makers get together in their communities to collaborate on projects for social change. Our Makers Making Change team will be part of WCD events in Vancouver, Regina, and Toronto. Specifically, we’ll be at Vancouver Hack SpaceCrashBang Labs, and Ryerson University. We are quite excited about WCD kicking off the first-ever Makers Making Change event in Saskatchewan! 2018 World Create Day Don’t live in one of these cities? You can search this Hackaday map for meetups in your area – or even organize your own. Either way, it’s a fantastic opportunity to collaborate and engage with makers in your community. We hope to see you there!

  • Meet the Makers Making Change Project Manager

    Chad03/01/2018 at 19:48 0 comments

    We are pleased to introduce Zee Kesler, our Makers Making Change Project Manager. Zee calls herself a Maker Educator. She has been an active participant in maker educator spaces since the first Vancouver Mini Maker Faire in 2011.

    Zee Kesler, Makers Making Change Project Manager

    Zee recently found out that she has a connective tissue disorder called Ehlers-Danlos Syndrome. She decided to get involved with Makers Making Change as a way to get more comfortable with having a disability, but also because she loves "maker culture, crowd-sourcing ideas, creating community, and being involved in innovative social programming."

    Zee inspecting the LipSync's main interface board

    Zee at the Bell LipSync Buildathon in October 2017

    Zee has co-founded/founded two mobile classrooms: the MakerMobile, which is a mobile hackspace, and the Magic Trout Imaginarium, which is a mobile classroom in a tiny house.

    Zee has been with us since September 2017, but it's still a good time to say we are thrilled to have her on board!

  • The LipSync is Featured on CTV

    Chad02/22/2018 at 18:26 0 comments

    On Monday, February 19th, CTV's Your Morning program aired a segment on assistive technology. Host Ben Mulroney and tech expert Avery Swartz discussed the LipSync, along with some other assistive devices.

    Click on the image below to watch the video:

    Ben Mulroney and Avery Swartz on Your Morning

    It's great to see more and more people talking about accessibility and assistive solutions in the mainstream.

  • Building LipSyncs with Neil Squire Solutions

    Chad02/15/2018 at 19:34 0 comments

    In late January, the Neil Squire Solutions team worked with us to build some LipSyncs. Many of them were soldering for the first time, but they successfully built seven devices together. Check out some photos here.

    If you're new to making, you may shy away from taking on a project like the LipSync. To prove that it shouldn't be intimidating at all, we put together this time-lapse of one of the Solutions team members building a device. Click the image below to open the video in a new window:

    Two hands working on LipSync parts

    On February 3rd, Piriya and Gemma from the Solutions team were at the ATIA Maker Day in Orlando. They saw several interesting projects and learned a lot:

    3D-printed tilt-compensating wheelchair cup holder

    Above: A 3D-printed tilt-compensating wheelchair cup holder

    Piriya learning to build an inexpensive phone stand made of corrugated plastic

    Above: Piriya learning to build an inexpensive phone stand made of corrugated plastic

    For more coverage, check out this article from, the hosts of the ATIA Maker Day.

  • 85 Students Attended the Acadia University LipSync Buildathon

    Chad02/08/2018 at 23:02 0 comments

    We had a wonderful time at the Acadia University LipSync Buildathon on February 1st. It was a day of teamwork and learning as scores of students came together to build 36 LipSyncs.

    Around 60 engineering students participated in building the LipSyncs. Of them, over 30 volunteered their time to help others build, including mentoring 25 Wolfville School students from Grades 7 and 8.

    Group photo of the attendees of the Acadia University LipSync Buildathon

    Above: What an amazing turnout!

    Acadia University’s engineering department incorporated the LipSync Buildathon into their classes, using the device to teach students about soldering, 3D-printing, electrical components, and accessibility.

    Students working on LipSync parts

    Above: A team of students working on LipSync parts

    Dwayne, who has quadriplegia, works at the Kings Regional Rehabilitation Centre as IT support. He spoke to the students about the importance of assistive technology. Dwayne is looking forward to using the LipSync and providing the device to residents who may find it useful.

    A student focusing on soldering

    Above: A student focusing on soldering

    For more in-depth coverage of the Buildathon, check out this article from the Annapolis County Spectator.

    Kudos to these brilliant students who did such a great job grasping the technology and helping people who need it!

  • ATIA Maker Day

    Chad02/01/2018 at 23:29 0 comments

    The ATIA conference has been on for nearly 20 years, but this is the first time it includes an ATIA Maker Day. It’s a great opportunity for both newcomers and seasoned makers to dive into the maker community.

    The ATIA Maker Day is powered by, which is run by Bill Binko. Bill is a phenomenal maker – we’ve even 3D-printed some of his designs in our lab.

    Click on the image below to watch a video introducing the ATIA Maker Day:

    A screen capture of the ATIA Maker Day video featuring Bill Binko

    Drop by if you are able and learn about soldering, 3D-printing, switches, and a lot more (it's free)!

    When: February 3rd, 9:00am to noon

    Where: Caribe Royale Hotel and Convention Center, Orlando, Florida

  • Acadia University LipSync Buildathon

    Chad01/25/2018 at 19:25 0 comments

    On February 1st, we will be in Wolfville, Nova Scotia for the Acadia University LipSync Buildathon. This the first-ever LipSync Buildathon in Atlantic Canada, so we're quite excited.

    A student working on a LipSync

    A West Island College student working on a LipSync during our buildathon with STEM Learning Lab in Calgary in December

    Around 20 Acadia University students will build LipSyncs for people with disabilities as part of Accessibility Week at the university. Learn more and register on the Eventbrite page.

    When: Thursday, February 1st, 9am - 4pm

    Where: Acadia University, Sheldon L. Fountain Learning Commons, 26 Crowell Road, Wolfville, Nova Scotia B4P 2R6

  • The Year in Review

    Chad12/21/2017 at 20:15 0 comments

    What a fantastic year it has been. We started off with the Access Makeathon, a 48-hour challenge where Makers worked together to create innovative assistive technologies for people with disabilities. And then we spent the entire year hopping around North America to meet dedicated Makers and build devices.

    10-year-old Timothy

    Above: Timothy, a 10-year-old with an anoxic brain injury, tests out an accessible Wii controller that a team built for him at the Access Makeathon

    We hosted or co-hosted 17 events, of which five were in the United States. With the help of many bright Makers, students, engineers, and tinkerers, we built 320 LipSyncs this year!

    Jim tests out a LipSync

    Above: Jim, who has a spinal cord injury, tests out a LipSync that students made for him at a Burnaby School District Makeathon

    Google and Neil Squire Society staff pose in front of the Android statue at Google HQ

    Above: Google and Neil Squire Society staff at Google headquarters for a buildathon during Accessibility Week

    Thank you to everyone for their skills, enthusiasm, and dedication towards making this happen. If you want to help in other ways, please consider making a donation this holiday season to support people with disabilities. Happy holidays and a wonderful new year!

  • The Youngest Person to Build a LipSync

    Chad12/15/2017 at 00:14 0 comments

    We were in Calgary last weekend for a buildathon with STEM Learning Lab. Around 25 West Island College students attended to help out, of whom only three had soldered before. But they were such brilliantly quick learners that by the end of the day, they had built 17 LipSyncs together.

    Students building LipSyncs

    Above: A group of students building LipSyncs

    Although the students worked in teams, an 11-year-old named Isabel built a device all by herself! She’s the youngest person yet to build a LipSync.

    Chad Leaman with Isabel

    Above: Chad Leaman, our Director of Innovation, with Isabel, the 11-year-old whiz

    Our speaker at the event was Mike, who has amyotrophic lateral sclerosis (ALS, also called Lou Gehrig’s disease). He has been inventing a range of devices to help him increase his independence.

    Mike brought with him his newest prototype, which is designed as a forklift for plates. It allows him to pick up a plate, put it into a microwave, take it out, and then bring it to his foot-controlled feeding device.

    Mike, Maya and the prototype

    Above: Mike with his daughter, Maya, alongside his prototype

    For more photos, please view our Facebook album.

  • We Fixed 67 LipSyncs

    Chad12/07/2017 at 22:20 0 comments

    On Monday, December 4th, we were at Vancouver Hack Space to repair a few LipSyncs. “A few” turned out to be 67: that’s how many LipSyncs are up and running now with the help of some fantastic Makers, all in 10 hours.

    The completed LipSyncs are on their way to people who need them. A big thank you to everyone who came out to troubleshoot with us!

    Makers working on LipSync parts

    Above: Makers putting their skills to good use. (Background, left) Luke, a Director at Vancouver Hack Space; (background, right) Marcello, a TELUS team member who also attended a previous buildathon; (foreground, left) Martin, an electrical engineer; (foreground, right) Charles, Mechanical Engineer at the Neil Squire Society

    Luke working on a LipSync

    Above: Luke working on LipSync parts

View all 50 project logs

Enjoy this project?



Mark Brady Ingle wrote 07/02/2017 at 22:52 point

is anyone still working on this project?  I need some help and ordering information on the mouth piece parts

  Are you sure? yes | no

Chad wrote 07/08/2017 at 16:14 point

hey Mark glad we caight up on email..sorry not logging into hackaday as often as I should. 

  Are you sure? yes | no

Mark Brady Ingle wrote 06/27/2017 at 02:50 point

I am testing on Mac and need the instructions for the RED/Green flash sequence.  I have a bluetooth connection established but I am not able to go any further.  Help would be appreci

  Are you sure? yes | no

Trent Robertson wrote 05/30/2017 at 16:00 point

Thank you for sharing this project. One of my best friends has ALS and would benefit from this. I have already set up voice recognition features in her apartment to allow her to control lights, air conditioning, and the TV, but she still needs someone to hold and control a tablet for her if she wants to surf the internet. I hope to find the time to build this soon.

  Are you sure? yes | no

Chad wrote 07/08/2017 at 16:13 point

hey Trent let us know if we can send you some parts / the main boards!

  Are you sure? yes | no

Mark Brady Ingle wrote 05/09/2017 at 20:53 point

Hello team!  Are the current instructions online the most current?  I have enough parts to get started so I wanted to make sure. 

  Are you sure? yes | no

Chad wrote 05/11/2017 at 20:20 point

Hi Mark, we have added the most recent as of now.  Let us know if you got questions with the build and in particular ping @Ivan.Gee if you need in particular details

  Are you sure? yes | no

eroy wrote 01/02/2017 at 23:34 point

Thank you for this project! I am endlessly disappointed in the cost of assistive technology. To make something like this open source is a gesture rarely seen. 

  Are you sure? yes | no

Charles G wrote 02/26/2017 at 06:49 point

Thank you for the kind words eroy! Stay tuned in the next couple weeks as we launch a new and improved version of the LipSync at an even lower price point!

  Are you sure? yes | no

Manuel Z wrote 10/10/2016 at 19:18 point

this is an amazing project! congratulations!!!! 

  Are you sure? yes | no

Charles G wrote 10/10/2016 at 23:14 point

Thanks, Manuel!

  Are you sure? yes | no

Sadiq Mohamed wrote 09/22/2016 at 12:02 point

A very worthwhile project. I had a friend with MS who would have benefited from this. Unfortunately he died some years ago. I wish you the best of luck with your testing and look forward to the results.

  Are you sure? yes | no

Ivan wrote 09/22/2016 at 17:55 point

My condolences about your friend @Sadiq Mohamed . I am glad you believe in our project, we are working our tails off over here so we can start getting makers and users together to start building these and help increase people's quality on life. Just as a side note, our plans are to continue compiling and refining open-source assistive technologies, if you have any ideas I would love to hear them.

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates