close-circle
Close
0%
0%

LipSync

An assistive tech which allows quadriplegics to use touchscreen mobile devices using a mouth-operated joystick with sip and puff controls

Similar projects worth following
close
Touchscreen devices have become a staple piece of technology in this day and age. Most people cannot get through the day without using their smartphone. It keeps us connected. It keeps us organized. It is an integral part of our social lives. For people in wheelchairs whom experience difficulties with fine upper body motor control, the usage of mobile devices can be very challenging. Statistics show there are over 1 million people in the United States and Canada that have very limited or no use of their hands, making touchscreen devices very difficult or impossible to use.

The "LipSync" is an assistive technology device which is being developed to allow quadriplegics and other people with limited hand use the ability to use touchscreen mobile devices by manipulation of a mouth-operated joystick with integrated sip and puff controls. We are releasing all of our work open-source, to make the Lipsync a solution that can be made at the community level for less than $300.

OVERVIEW

In 2016, upwards of 1.6 billion people in the world are using smartphones. Smartphones are becoming a staple piece of technology for many citizens in North America and around the World. Smartphones and touchscreen devices enable users to navigate around their city better, to communicate with others more freely, and to operate applications which can promote happy and productive lives. Currently, there exist a limited number of practical devices for quadriplegics to use touchscreen devices - this is where we step in.

The LipSync is an electronic device which allows quadriplegics the ability to use compatible touchscreen and computer devices without the use of their hands. The user is able to manipulate a cursor on their device screen using a mouth-operated joystick with integrated sip and puff controls to simulate the actions of "tap" and hitting the back button, respectively. With longer sips and longer puffs, additional secondary features are enabled including a "tap and drag", "long tap and drag" and the possibility of more specialized functions as per the user's needs.

The LipSync is design specifically for portable devices, it does not require AC power, but it will work with any device including desktop and laptop computers that support mice through a universal serial bus (USB) or the Bluetooth connection.

The LipSync is an open-source hardware project where all of our 3D printer files, component lists and microcontroller code are made public. In the spirit of accessibility, our housing can be 3D printed, the electronic components are readily available and the assembly is as straightforward as possible.

The LipSync was envisioned as a holistic solution that takes in to consideration not only the interface but how the system is to be mounted on the user’s wheelchair. The actual electronics of the device is on part of the implementation. There are no standardized methods to which wheelchairs are designed. Wheelchair manufacturers can use round or square tubing. They often also use tubing which is not compatible with other manufacturers so that customers must buy accessories from them. As a result, there is not a standard location or clamping mechanism to mount assistive technology on the chair.

Wheelchairs are also customized to the user, including the height, width and seating position of the user. The seating on the wheelchairs are customized to minimize the incidents of pressure sores. As a result, the mounting system for assistive technology such as the LipSync also has to be customized. In addition to the instructions to assemble a LipSync, we have included instructions on how to mount a LipSync. A combination of off-theshelf and custom 3D printed components are provided in order to help makers create a fully integrated and customized solution for the user.

Now the maker community and disability community can meet, collaborate and work together constructing a LipSync over a period not much longer than a weekend. We hope these new relationships will continue spurring innovation within the maker community.

There are 3 main aspects we will be addressing in our project:

  1. Developing an easy to build, but robust electronics assembly that novice to experienced makers can build;
  2. Developing a device housing which can be 3D printed by makers either at home or at dedicated facilities;
  3. Creating mounting options for a variety of wheelchairs with 3D printed parts and/or commercially available components.

SMARTPHONES AND ACCESSIBILITY

Smartphones, by their very nature, are intended to be used while on the go. Traditional assistive technology designs for the desktop and laptop computer are not portable in nature, so they cannot be easily applied to smartphones. While single and dual input systems exist for smartphones, they are slow and frustrating to use for users with more capabilities of movement. Single and dual input switches are appropriate for users that can only make one or two...

Read more »

LipSync_Manual_v1.4.2.pdf

Instructions for assembling a LipSync

Adobe Portable Document Format - 10.55 MB - 08/21/2017 at 17:18

eye
Preview
download-circle
Download

LipSync_Startup_Guide_v1.23.pdf

Startup guide for setting up and using a LipSync

Adobe Portable Document Format - 1.78 MB - 07/13/2017 at 18:17

eye
Preview
download-circle
Download

LipSync_STL_v2.2d.zip

Open README file for print settings before printing!!! STL folder contains all necessary print files.

Zip Archive - 1003.74 kB - 06/21/2017 at 20:36

download-circle
Download

LipSync_Firmware.ino

LipSync Firmware Version Version 2.6 ( 9 May 2017 )

ino - 39.29 kB - 06/08/2017 at 01:07

download-circle
Download

LipSync_BOM.xlsx

LipSync Components in Excel format

sheet - 15.74 kB - 08/04/2017 at 03:42

download-circle
Download

View all 12 files

View all 38 components

  • TOM:Calgary LipSync Buildathon

    Chad08/24/2017 at 17:36 0 comments

    Greetings from Calgary! We’re here for the 2017 TOM:Calgary Makeathon. In addition to being a headline sponsor of the event this weekend, we’re happy to be partnering with TOM:Alberta to host our first LipSync Buildathon in Alberta — hopefully the first of many!

    Our engineer, Charles, helping high school students at a previous Buildathon.

    Our engineer, Charles, helping high school students at a previous Buildathon.

    The TOM:Calgary Makeathon is a three-day event running Friday to Sunday (August 25th -27th) at 3536-27 Street NE, Calgary. Similar to our Access Makeathon, 15 teams of 4-6 individuals each will work with a person with a disability to create a solution to an everyday barrier they have. Participants will be given a budget, monitored, and guided through their projects. On Sunday, the projects will be showcased from 2:30-5:30 PM at the University of Calgary EEEL building. If you’re in the area, you can RSVP at no cost, by visiting bit.ly/yycdemoday.

    The LipSync Buildathon will take place on the Friday and Saturday at the same place from 12-6 PM. High school students and recent undergraduates will be working with technical mentors from TOM:Alberta and Makers Making Change to build over 15 LipSync devices which will be given to quadriplegics living in Southern Alberta.

    PS We have entered the LipSync into the Instructables Makerspace Contest. If we win, the prizes are 3D printers, a Laser cutter and a ton of other really useful equipment that will help us grow Makers Making Change. Feel free to help us win by signing up for an account (if you don’t have one already), and clicking vote on the three contest options: https://www.instructables.com/id/How-to-Build-a-LipSync/

  • AMI Segment

    Chad08/10/2017 at 16:10 0 comments

    Accessible Media Inc. (AMI) put together a great feature on our LipSync Buildathons. Have a look, it’s right at the beginning of their weekly show.

    a picture of the LipSync, as featured on AMI This Week

    Though our segment is at the beginning, we’d recommend you should watch the whole episode — they profile a whole bunch of cool accessibility initiatives across Canada.

    As Fall approaches, we’ve hosted eight Buildathons across Canada and the US, and are finalizing a number of Buildathons which we've lined up for the coming months of 2017. But we’re still looking to do more. If you or your organization is interested in hosting or helping fund your own LipSync Buildathon, please feel free to contact chadl@neilsquire.ca.

  • Burnaby South Secondary Volunteers

    Chad08/08/2017 at 17:16 0 comments

    Over the last few weeks, we’ve been hosting a few very welcome guests. A number of Burnaby South Secondary students have been coming to our office a few times a week to build LipSyncs.

    Marcus (left) and Sam (right) building LipSyncs

    Marcus (left) and Sam (right) have been among the Burnaby South Secondary students coming to the Neil Squire Society’s Burnaby office to build LipSyncs.

    As you may remember, Burnaby South Secondary is where the first two of our high school Buildathons took place. Sam, who just finished Grade 11, was one of the makers who was at both. The makers at the events got to see first firsthand people receiving the LipSyncs they had just built. For Sam, that experience was quite profound, seeing someone benefit from his work, and be able to use a smartphone — something that most people take for granted.

    “It was pretty cool,” Sam explains. “It really put a smile on my face.”

    Looking to volunteer, Sam decided to do something he both found fun — building a LipSync wasn’t the first time he soldered — and something he knew made a difference. He could put his skills to good use.

    “Not only am I doing things that I’m familiar with, but also [it’s about] what it is for,” Sam says.

    That’s truly what the LipSync project — and on a larger level, Makers Making Change — is all about. We want to facilitate a kind of skilled philanthropy, helping people use their skills to create access for people with disabilities.

  • Ashley's Story

    Chad07/27/2017 at 17:25 0 comments

    We have another great Access Makeathon story to show. While we’ve previewed Ashley’s story at a handful of events, this is the first time it’s available publicly. We think that this story is great in that it shows how Ashley was an active participant in the process — it was very much a collaborative effort between her and the makers. It also shows the wide range of things, whether it seems big or small, that could really improve a person’s life — that a maker could make. Have a look here:

    Thank you to Ashley for participating in this video and taking such an interest in the Makers Making Change initiative. 

    We are looking forward to doing more open-style Maker events in the coming months. With that in mind, we are happy to announce that we will be a headline sponsor of the Tikkun Olam Makers (TOM): Alberta Makeathon 2017 from August 24th - 27th in Calgary. We had initially come into contact with TOM, who are a global movement that put together Makeathons similar to our Access Makeathon, in San Francisco earlier this year, and are happy to be working with them in Calgary. As part of the Makeathon, there will be a LipSync Buildathon. We are hoping this is a successful first step into Alberta.

  • Tacoma Buildathon

    Chad07/20/2017 at 16:57 0 comments

    We have another Stateside Buildathon coming up, and this one’s close to home — we’ll be in the Pacific Northwest just outside of Seattle in beautiful Tacoma, Washington on Monday, July 31st.We’re starting to get quite well acquainted with the Pacific Northwest.

    This Buildathon will be a little different, however. We’ll be seeing a lot of Washington-based educators, who can receive continuing education credits from Pacific Lutheran University for attending this event. FabLab Tacoma, who we partnered with to bring a Buildathon to Tacoma and are hosting the event, saw this as an opportunity for educators to learn about Buildathons and maker culture, and potentially add them to their curriculums. We’re hoping this leads to more Buildathons involving students in Washington, like the ones we’ve done in Burnaby.

    For more information and to register, go to: www.fablabtacoma.com/event-2582186.

  • Jim and Isabelle's Story

    Chad07/13/2017 at 17:47 0 comments

    Have you seen Jim and Isabelle’s story? Filmed during our Access Makeathon, it’s a great example of how much of a difference the maker movement can make for people with disabilities — it’s why Makers Making Change is so important. Have a look here:

    Thank you to Jim and Isabelle for participating in this video and taking such an interest in the Makers Making Change initiative. We’ve received invaluable feedback from Jim on the LipSync.

    Stay tuned, we’ll have another Access Makeathon success story rolling out in the next few weeks.

  • RESNA Award and Thanks

    Chad07/06/2017 at 17:15 0 comments

    We had a great time at the 2017 RESNA (Rehabilitation Engineering and Assistive Technology Society of North America) Annual Conference. Our Manager of Research and Development, Harry Lew, presented his paper on the Makers Making Change initiative. To our honour, we ended up taking home the Most Disruptive Technology award.

    Our engineer, Charles, standing with Harry (left to right) at our booth at RESNA.

    It’s been just over a year since we began building our LipSync team, for the major project which led into our new Makers Making Change initiative. We’ve made a lot of progress in such a short amount of time, and have been blown away with the amount of interest in our work. With over 120 LipSyncs built and ready to ship out, a new website in development, and work being done on investigating and supporting other open-source assistive technology solutions, this is just the beginning.

    All of this work comes from the staff at the Neil Squire Society, who use technology, knowledge and passion to empower Canadians with disabilities. Our staff, interns and volunteers have put a tremendous amount of time and energy into the Makers Making Change initiative, yet our success wouldn’t have been possible without the help of some key supporting organizations, and we’d like to thank them now.

    Google.org’s grant allowed us to take the LipSync from a prototype on a shelf to actually having an open-source, low cost “mouse in your mouth” solution. It is already being made and used in communities across North America, with many of you asking to get involved.

    The Vancouver Foundation was the first to fund Maker Making Change in British Columbia, which allowed us to really create the branding, host our first Access Makeathon, and begin exploring other open source assistive technologies.

    Recently, the Government of Canada announced funding over the next two years, which will really allow us to bring this initiative across Canada, connecting makers to people with disabilities who need assistive technologies. We are all so incredibly excited about this opportunity to scale this project to Neil Squire Society offices across Canada through this support.

    Finally, and most importantly, we’d like to thank you. We’ve spent a lot of time doing outreach over the last few months, and we’ve been delighted by the interest across Canada and the United States — overwhelmed, really. Our vision of open source assistive technology hardware solutions is bold, and we are grateful for your interest in helping us grow this network.

  • The CAOT Conference in Charlottetown

    Chad06/29/2017 at 18:40 0 comments

    Wow! June was one busy month. We capped off a month of Buildathons, Maker Faires, and outreach with a trip to Charlottetown, PEI (and the surrounding area) for the 2017 Canadian Association of Occupation Therapists (CAOT) Conference, similar to the American Occupational Therapist Association conference we attended in Philadelphia.

    Chad Leaman, Director of Development, standing in front of Confederation Bridge in Prince Edward Island.

    In Philadelphia, we connected with occupational therapists across the United States and Canada and we created connections and roots in the city and surrounding areas with local MakerSpaces, rehabilitation clinics, and interested people. It’s allowed us to host Buildathons and really be visible in Philadelphia — it paved the way for a lot of progress.

    At the CAOT conference in PEI, we aimed to replicate the outreach we achieved in Philadelphia. Highlights included meeting with and giving LipSyncs to the University of Prince Edward Islandand their disability services office, and Spinal Cord Injury Prince Edward Island. Rather coincidentally, Saturday was the PEI Walk for ALS, where we connected with a Maritimes service provider, Harding Medical, and gave them a LipSync.

    We also dropped off an unassembled LipSync kit with the Summerside MakerSpace which they are currently building. The MakerSpaces we met were very enthusiastic and willing to help.

    Above: Our Industrial Designer, Kristina, inspecting our 3D printer at the conference.

    Below: Chad Leaman with Dr. William Miller, a Professor in the Department of Occupational Science and Occupational Therapy and an associate dean at UBC.

    While we have talked mostly about our activities outside the conference, the conference itself was a huge success. Just as we had experienced in Philadelphia, occupational therapists were tremendously interested in the LipSync and Makers Making Change. We made good contact with an occupational therapist or school in literally every province of Canada — and sometimes more, we met many from Nova Scotia and Alberta for instance. It’s a great launching point.

    These conferences provide a very efficient way to grow our networks and grow the initiative, allowing us to meet people from all over the country, rather than having to go to each community individually to make connections. Not that we don’t enjoy travelling to each different community — we’ll be doing that in our follow-up with many of our new connections. (We are looking forward, however, to next year’s CAOT conference being in Vancouver — a huge opportunity).

    A group photo of our Bellingham build’s participants. (For more pictures, check out The Foundry MakerSpace on Facebook).

    Rounding out our week, we were also in Bellingham, Washington, co-ordinating another successful build. Thanks to The Foundry MakerSpace, our gracious hosts, we now have another 17 LipSyncs built.

    After the final LipSync build of the month, it’s nice to look back — we built nearly 90 LipSyncs in June alone. We’re getting close to the 150 we’ve aimed to build during our first phase. Now, it’s time to really start getting them out to people on a large scale. For us, July will be a lot calmer, at least in terms of events, so over the next few weeks we’re going to be doing some quality assurance and work on getting the LipSyncs out there.

    Our Director of Research and Development, Harry Lew, is presenting at the RESNA (Rehabilitation Engineering and Assistive Technology Society of North America) 2017 Annual Conference in New Orleans, Louisiana this week. Lew will be comparing different open source models, and how the Makers Making Change model incorporates the strengths and works to overcome the weaknesses of previous models.

  • Stateside Buildathons

    Chad06/22/2017 at 18:31 0 comments

    We hadn’t done a Buildathon in the United States yet. So why not do two of them on the same day in opposite ends of the country? That’s what we did Sunday, as part of the Nation of Makers’ Week of Making, with Buildathons in Philadelphia, Pennsylvania, and Seattle, Washington.

    It was fantastic.

    father_daughter

    Above: Seeing that both Buildathons fell on Father’s Day, it was only fitting that we had a father-daughter team. Scott, our friend from Comcast who built one on his own initiative, pictured with his daughter.

    Below: We’ve typically done red LipSyncs, but while we’re trying new things, why not switch it up with the colours.

    color

    In Philadelphia, where we partnered with NextFab (who hosted us at their Philadelphia location), we got to say hi (and build LipSyncs) with a few old friends. Scott from Comcast brought along his daughter, and they built a LipSync.

    As well, a few of the people who we had dropped off LipSyncs with last time actually built one themselves. Charles, from Inglis House, and Russ, from the Institute of Disabilities at Temple University, both got to build a LipSync and take them home. It was great not only to give them a LipSync, but show them how they can make it themselves.

    “[Chad Leaman], Ivan and the rest of the team did a phenomenal job putting together the Buildathon this past weekend,” Russ, who received four LipSyncs for the Pennsylvania's Initiative on Assistive Technology, wrote us in an email. “The LipSyncs produced will, no doubt, be put to good use in our Statewide AT lending library.”

    russ

    Above: Russ shaking hands with Chad Leaman, Director of Development, holding his certificate of completion. (Thanks to Marcella Barker of NextFab, who took this and the following photos of the Philadelphia Buildathon).

    Below: Charles soldering together his circuit board.

    One last photo of our group in Philadelphia.

    In Seattle, at the SoDo MakerSpace, we also had a great time. In all, we ended the weekend with 21 LipSyncs built.

    Above: The makers at the SoDo MakerSpace hard at work.

    Below: The result.

    Buildathons weren’t the only thing we were doing in the States. Our Executive Director, Gary Birch, was in Washington DC, as a panelist at the M-Enabling Summit. In addition to talking to a large audience of influential persons and industry leaders about the LipSync and Makers Making Change, Birch also met some important contacts – he paid a visit to MedStar National Rehabilitation Hospital, and met with John Anschutz, the manager of the Assistive Technology Center at the Shepherd Center, one of the best rehabilitation hospitals in the world. He gave them both LipSyncs.

    Gary Birch demonstrating the LipSync to John Anschutz

    Gary Birch demonstrating the LipSync to John Anschutz, the manager of the Assistive Technology Center at the Shepherd Center.

    Our travels are not over. We’re at the 2017 Canadian Association of Occupational Therapists conference right now. With the success of the American one in Philadelphia — the connections we made were why we were there on Sunday — we hope to find the same in Charlottetown, PEI. We’ll meet with occupational therapists from all over Canada, as well as some local makerspaces, and give out a few LipSyncs.

  • Vancouver Mini Maker Faire

    Chad06/15/2017 at 17:57 0 comments

    It’s good to be home — for now. Before heading out for a number of events on the road for the last few weeks of June, it was good to be at the Vancouver Mini Maker Faire right near home, and in our local community.

    Above: Engineer Charles and Director of Development, Chad Leaman, at our booth. (For more fun pictures of the Maker Faire, check out their photo album on Facebook).

    Below: Engineers Rashmi, Amanda, and Ivan talk to an interested spectator.

    It was great to talk to the community, getting the chance to meet with a whole of new faces.

    But it also gave us time to reconnect with some “old” friends. We got a chance to catch up with a lot of the people who helped make the Access Makeathon so successful. Many of the individual makers, and organizing partners including SFU SIAT, Vancouver Hack Space, RP Electronics were there at the PNE Forum with us Saturday and Sunday.

    In addition to the previously announced Philadelphia LipSync Buildathon this Sunday, we’ve added a couple of US Buildathons in the next few weeks as part of the Nation of MakersWeek of Making. Both are in Washington State. The first is in Seattle this Sunday (the same day as the Philadelphia one), while we have one in Bellingham on the following Saturday (the 24th).

    If you happen to be in the area for any of the three Buildathons mentioned — Philadelphia, Seattle, and Bellingham — they are still looking for both people to make LipSyncs, and people who need LipSyncs. The events are open to the public, so you can even just come and watch. Click the previous links on the city names for more information and to register.

View all 32 project logs

View all instructions

Enjoy this project?

Share

Discussions

Mark Brady Ingle wrote 07/02/2017 at 22:52 point

is anyone still working on this project?  I need some help and ordering information on the mouth piece parts

  Are you sure? yes | no

Chad wrote 07/08/2017 at 16:14 point

hey Mark glad we caight up on email..sorry not logging into hackaday as often as I should. 

  Are you sure? yes | no

Mark Brady Ingle wrote 06/27/2017 at 02:50 point

I am testing on Mac and need the instructions for the RED/Green flash sequence.  I have a bluetooth connection established but I am not able to go any further.  Help would be appreci

  Are you sure? yes | no

Trent Robertson wrote 05/30/2017 at 16:00 point

Thank you for sharing this project. One of my best friends has ALS and would benefit from this. I have already set up voice recognition features in her apartment to allow her to control lights, air conditioning, and the TV, but she still needs someone to hold and control a tablet for her if she wants to surf the internet. I hope to find the time to build this soon.

  Are you sure? yes | no

Chad wrote 07/08/2017 at 16:13 point

hey Trent let us know if we can send you some parts / the main boards!

  Are you sure? yes | no

Mark Brady Ingle wrote 05/09/2017 at 20:53 point

Hello team!  Are the current instructions online the most current?  I have enough parts to get started so I wanted to make sure. 

  Are you sure? yes | no

Chad wrote 05/11/2017 at 20:20 point

Hi Mark, we have added the most recent as of now.  Let us know if you got questions with the build and in particular ping @Ivan.Gee if you need in particular details

  Are you sure? yes | no

eroy wrote 01/02/2017 at 23:34 point

Thank you for this project! I am endlessly disappointed in the cost of assistive technology. To make something like this open source is a gesture rarely seen. 

  Are you sure? yes | no

Charles G wrote 02/26/2017 at 06:49 point

Thank you for the kind words eroy! Stay tuned in the next couple weeks as we launch a new and improved version of the LipSync at an even lower price point!

  Are you sure? yes | no

Manuel Z wrote 10/10/2016 at 19:18 point

this is an amazing project! congratulations!!!! 

  Are you sure? yes | no

Charles G wrote 10/10/2016 at 23:14 point

Thanks, Manuel!

  Are you sure? yes | no

Sadiq Mohamed wrote 09/22/2016 at 12:02 point

A very worthwhile project. I had a friend with MS who would have benefited from this. Unfortunately he died some years ago. I wish you the best of luck with your testing and look forward to the results.

  Are you sure? yes | no

Ivan wrote 09/22/2016 at 17:55 point

My condolences about your friend @Sadiq Mohamed . I am glad you believe in our project, we are working our tails off over here so we can start getting makers and users together to start building these and help increase people's quality on life. Just as a side note, our plans are to continue compiling and refining open-source assistive technologies, if you have any ideas I would love to hear them.

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates