Close
0%
0%

LipSync

An assistive tech which allows quadriplegics to use touchscreen mobile devices using a mouth-operated joystick with sip and puff controls

Similar projects worth following
Touchscreen devices have become a staple piece of technology in this day and age. Most people cannot get through the day without using their smartphone. It keeps us connected. It keeps us organized. It is an integral part of our social lives. For people in wheelchairs whom experience difficulties with fine upper body motor control, the usage of mobile devices can be very challenging. Statistics show there are over 1 million people in the United States and Canada that have very limited or no use of their hands, making touchscreen devices very difficult or impossible to use.

The "LipSync" is an assistive technology device which is being developed to allow quadriplegics and other people with limited hand use the ability to use touchscreen mobile devices by manipulation of a mouth-operated joystick with integrated sip and puff controls. We are releasing all of our work open-source, to make the Lipsync a solution that can be made at the community level for less than $300.

OVERVIEW

In 2016, upwards of 1.6 billion people in the world are using smartphones. Smartphones are becoming a staple piece of technology for many citizens in North America and around the World. Smartphones and touchscreen devices enable users to navigate around their city better, to communicate with others more freely, and to operate applications which can promote happy and productive lives. Currently, there exist a limited number of practical devices for quadriplegics to use touchscreen devices - this is where we step in.

The LipSync is an electronic device which allows quadriplegics the ability to use compatible touchscreen and computer devices without the use of their hands. The user is able to manipulate a cursor on their device screen using a mouth-operated joystick with integrated sip and puff controls to simulate the actions of "tap" and hitting the back button, respectively. With longer sips and longer puffs, additional secondary features are enabled including a "tap and drag", "long tap and drag" and the possibility of more specialized functions as per the user's needs.

The LipSync is design specifically for portable devices, it does not require AC power, but it will work with any device including desktop and laptop computers that support mice through a universal serial bus (USB) or the Bluetooth connection.

The LipSync is an open-source hardware project where all of our 3D printer files, component lists and microcontroller code are made public. In the spirit of accessibility, our housing can be 3D printed, the electronic components are readily available and the assembly is as straightforward as possible.

The LipSync was envisioned as a holistic solution that takes in to consideration not only the interface but how the system is to be mounted on the user’s wheelchair. The actual electronics of the device is on part of the implementation. There are no standardized methods to which wheelchairs are designed. Wheelchair manufacturers can use round or square tubing. They often also use tubing which is not compatible with other manufacturers so that customers must buy accessories from them. As a result, there is not a standard location or clamping mechanism to mount assistive technology on the chair.

Wheelchairs are also customized to the user, including the height, width and seating position of the user. The seating on the wheelchairs are customized to minimize the incidents of pressure sores. As a result, the mounting system for assistive technology such as the LipSync also has to be customized. In addition to the instructions to assemble a LipSync, we have included instructions on how to mount a LipSync. A combination of off-theshelf and custom 3D printed components are provided in order to help makers create a fully integrated and customized solution for the user.

Now the maker community and disability community can meet, collaborate and work together constructing a LipSync over a period not much longer than a weekend. We hope these new relationships will continue spurring innovation within the maker community.

There are 3 main aspects we will be addressing in our project:

  1. Developing an easy to build, but robust electronics assembly that novice to experienced makers can build;
  2. Developing a device housing which can be 3D printed by makers either at home or at dedicated facilities;
  3. Creating mounting options for a variety of wheelchairs with 3D printed parts and/or commercially available components.

SMARTPHONES AND ACCESSIBILITY

Smartphones, by their very nature, are intended to be used while on the go. Traditional assistive technology designs for the desktop and laptop computer are not portable in nature, so they cannot be easily applied to smartphones. While single and dual input systems exist for smartphones, they are slow and frustrating to use for users with more capabilities of movement. Single and dual input switches are appropriate for users that can only make one or two movements consistently. The LipSync is...

Read more »

  • What AT Can Help Students with Disabilities?

    Chad3 days ago 0 comments

    We are currently working on expanding our open-source assistive technology library by adding devices that can help youth. We have some questions for teachers to help us get started.

    A hand using a key turner to open a door

    The key turner, a popular low-tech AT device from our library

    What disabilities do your students have? Are there any assistive devices you can think of that would be a valuable addition to your classroom? If your students are already working with some assistive technology, click a photo of the device and send it to us!

    You can reach out by contacting Zee Kesler, Makers Making Change Project Manager, at zeek@neilsquire.ca. Thank you for your help!

  • We Fixed 40 LipSyncs

    Chad6 days ago 0 comments

    On Thursday, July 5th, Makers Making Change was at Vancouver Hack Space to repair a few LipSyncs. In just a few hours, around 12 volunteers helped us troubleshoot and fix 40 LipSyncs. We can now send these completed LipSyncs to people with disabilities who need them. Thank you to everyone who came out to help! Staff and volunteers at a table Above: Makers Making Change staff and volunteer makers working on LipSync parts Shaemus, our Research and Development student, soldering Above: Our Research and Development student, Shaemus, soldering Completed LipSyncs after the event Above: A whole bunch of completed multi-coloured LipSyncs

  • Dementia-Friendly Music Player

    Chad07/09/2018 at 21:16 0 comments

    At Makers Making Change, we believe in collaborating to make even more effective assistive solutions. On that note, we want to introduce two makers: Ross Porter, a Seattle-based maker who created the dementia-friendly music player, and Trey Bagley, who made the 3D-printed design.

    The laser-cut design of the music player

    The laser-cut design of the music player

    This music player was Ross’s first maker project. He set out to create something that would bring joy to his father, who had dementia. The project is completely open-source, with the initial model consisting of a laser-cut case.

    “My dad could no longer operate a CD player or iPod,” says Ross on his website, DQMusicBox.org. “But he could use the music player that I designed and made for him, because it operates like a familiar two-knob radio. I was inspired to make this by the documentary Alive Inside which shows the profound joy felt by some people with dementia when listening to their favorite music.”

    The 3D-Printed Design

    The stained 3D-printed cathedral-style design

    The stained 3D-printed cathedral-style design

    Ross was now looking for a 3D-printed version of the case, so that the project would be more accessible to potential makers. We connected him to maker Trey Bagley, who worked with Ross to create an elegant cathedral-style shell. The filament is made of 40% sawdust and can be stained to make it appear closer to wood. The design stays just as simple to use.

    “If you imagine someone with dementia, they gradually forget the present, but their old memories are strong,” explains Ross. “Particularly, memories formed before the age of 21. Thus, Trey went about creating something that someone, even if their memory does not expand past the 1950s, would still recognize – something that they have experienced as a child.”

    “I thought it was a very interesting design challenge,” says Trey. “We did a lot of research, pulled up some specific examples, mostly from people on EBay selling antiques and vintage collectibles. The wood filament and the staining brought out a ton of detail. The filament can replicate the wood grain on a flat surface.”

    An Easy and Fun Project

    Components required to build a music box

    These are all the parts you need. Get making!

    At the Seattle Mini Maker Faire in September 2017, Ross provided some participants the required materials to build a device then and there. “I did get seven people to sign up and make the device – and all seven succeeded in making it. It takes an hour and fifteen minutes on average.”

    Do you know anyone in your life or in your community who could benefit from this music player? You can easily order the parts online and get either a laser-cut or 3D-printed shell. As Ross says, a 13-year-old can build it!

    You can find the instructions for both models or request the project on our website. If you have questions or suggestions about the music player, contact Ross at rosswesleyporter@gmail.com. If you have ideas about the existing 3D-printed shell or find any errors, contact Trey at tbornottb@sbcglobal.net.

  • Fix a LipSync

    Chad06/28/2018 at 17:54 0 comments

    Next week, we’ll meet our old friends at Vancouver Hack Space (VHS) to repair and complete a bunch of LipSyncs. Not every device is finished at our buildathons, and we periodically organize fix-it nights so community members can help us complete them.

    By volunteering to troubleshoot a LipSync, you help a person with a disability (who may be quadriplegic or have limited use of their hands) gain access to a touchscreen device. Plus: around 6pm, we'll get pizza for our volunteers.

    Luke working on a LipSync

    Luke, a director at VHS, during a Fix a LipSync event in December 2017. We had fixed 67 LipSyncs at that time!

    RSVP on Eventbrite. Come any time, for as long as you are able.

    When: Thursday, July 5th, 1pm - 8pm
    Where: Vancouver Hack Space, #104-1715 Cook St Vancouver, BC V5Y 3J6

    PS: Do you have a chronic condition/disability or care for someone who does? The UBC Health Mentors Program is looking for Health Mentors to share their medical expertise with healthcare students. Mentors meet students twice a semester over a 16-month term, and share their expertise on living with a chronic condition.

    Applications are open till August 13th, 2018. Learn more about being a Health Mentor on the UBC website.

  • TELUS Staff Build 175 LipSyncs

    Chad06/21/2018 at 21:52 0 comments

    Last week, we wrapped up our TELUS Days of Giving LipSync Buildathons. Some incredible TELUS team members helped us build 175 LipSyncs over the past month and a half!

    These LipSyncs are available for free in British Columbia, Alberta, and Ontario. If you live in these provinces and are waiting on or want to trial a LipSync, please get in touch! Email Zee at zeek@neilsquire.ca and she’ll help you out.

    Here are a few photos recapping our buildathons. Check out our Facebook album for more photos.

    Group photo at Calgary buildathon

    Smiles and colourful LipSyncs at the Calgary buildathon on May 4th

    TELUS maker holding up a LipSync component

    TELUS maker at the Okanagan buildathon on May 12th

    Derrick working with maker

    Derrick working with a TELUS team member at the #BCTECH Summit on May 14th

    Thank you, TELUS, for your leadership and support in hosting these LipSync Buildathons, enabling us to create assistive solutions for people with disabilities all over Canada.

    PS: We’re at the CAOT Conference all week. Zee hosted a pre-conference workshop yesterday introducing maker skills to occupational therapists. It focused on using 3D modelling, 3D printing, and creating switch adapted toys. Here’s a peek at one such switch adapted toy.

  • A Better LipSync Joystick Jig

    Chad06/14/2018 at 18:32 0 comments

    One of the most challenging and error-prone steps in building the LipSync has been assembling the force-sensing resistors (FSRs). The delicate FSRs can be easily damaged by people who have never worked with them before.

    The most common problem during Makers Making Change build events was kinked or overheated FSRs. Multiply that by four FSRs for each LipSync and it doesn’t look great, especially since it’s the most expensive component.

    True story: Jake, now our Mechanical Engineer, accidentally destroyed two FSRs when he was volunteering during our first Buildathon.

    The old and new jig next to each other

    The old joystick jig in white (left) and the new one in black (right)

    Enter the new joystick jig, which helps makers install each FSR correctly onto the board. The notched slot holds each FSR in place as you solder and ensures correct alignment.

    Since we debuted this design, the number of damaged FSRs has reduced dramatically – also reducing the frustration associated with it. This means we can speed up our build time, so it’s a win-win all around.

    The colour-changing jig, currently purple

    Justin, a local maker from Hedgehog Technologies, made this cool colour-changing version of the joystick jig

    This is a testament to how 3D printing tools can make work more efficient. It’s relatively quick and easy to improve upon your design and make more effective solutions.

  • NOMCON and AT Makers' Fair

    Chad06/07/2018 at 17:41 0 comments

    We are rolling up our sleeves and prepping for a week of activity. Here are two big maker events in the US that we’ll be a part of.

    NOMCON (June 9 - 10)

    Nation of Makers logo

    This weekend, Chad Leaman and Harry Lew will be at the first-ever Nation of Makers Conference (NOMCON) down in Santa Fe. NOMCON is an unconference, meaning it focuses more on dialogue and connections than one-way presentations.

    Makers from all over the United States will be attending NOMCON. In fact, Harry and Chad will probably be the only Canadians there. So if you’re curious to meet some living, breathing Canadians, just watch out for Chad; he'll be the only person in a Team Canada jersey.

    On June 9th, Chad will be facilitating a conversation between brilliant makers who are focused on and interested in creating open assistive devices. Their discussion will also touch on intentional inclusion, which is the theme of the conference.

    When: June 9th, 11am

    Where: Room Nambe, Santa Fe Convention Center

    NOMCON has an all-star lineup of maker leaders and there's a lot to learn. We are looking forward to it! Here is some more info on NOMCON.

    AT Makers' Fair (September 29)

    Assistive Technology Makers' Fair logo

    Makers Making Change is on the planning committee for the Assistive Technology Makers’ Fair coming up on September 29th, 2018 in Concord, New Hampshire. The Fair will bring together over 200 makers, people with disabilities, and educators in the community.

    The keynote presenters are Therese Willkomm and Bill Binko, two front-runners in the AT maker movement. The Fair is currently looking for sponsors, booths, and attendees, so we hope to see you there! Register here for the AT Makers' Fair.

    PS: On May 30th in the House of Commons, Prime Minister Justin Trudeau reinforced the Government of Canada’s commitment to help more Canadians with disabilities participate in the global economy, saying:

    “… As the world becomes increasingly digital, we want to ensure all Canadians are set up for success. In Vancouver, we announced $3 million for the Neil Squire Society to develop an open-source assistive technology model across Canada that will greatly reduce costs for persons with disabilities. This is part of our $22.3 million accessible technology program, which will help overcome barriers and give all Canadians an equal opportunity to obtain good jobs for today and tomorrow.” Read the full transcript here.

  • New Makers Making Change Website

    Chad05/31/2018 at 21:20 0 comments

    We are thrilled to officially invite you to the shiny new Makers Making Change website: www.makersmakingchange.com. Our goal is to facilitate more community level collaboration, which is why the homepage will greet you with a feature allowing you to search for makers, project requests, or events in your area.

    The new Makers Making Change website homepage

    The new Makers Making Change website homepage

    If you navigate to the Projects section of the website, you’ll find some projects uploaded by makers and marked as ideas, prototypes, or complete. You can contribute your skills to work on projects in progress. If you have a disability and feel a certain project will improve your life, you can request a build.

    Our hope is that more people sign up and share projects that can help people with disabilities. We encourage you to spend some time on the website, browse projects, and connect with some amazing people.

  • Government of Canada Funding

    Chad05/31/2018 at 21:10 0 comments

    On Global Accessibility Awareness Day, the Government of Canada stated its commitment to help more Canadians with disabilities participate in the digital economy. 

    More than $3 million in funding for the Makers Making Change initiative was announced by the Honourable Navdeep Bains, Minister of Innovation, Science and Economic Development, and the Honourable Kirsty Duncan, Minister of Science and Minister of Sport and Persons with Disabilities, at the Rick Hansen Institute (RHI), a Vancouver-based not-for-profit organization that drives innovation in spinal cord injury research and care.

    Global Accessibility Awareness Day panel members

    Left to right: Dr. Gary Birch, Executive Director of the Neil Squire Society; the Honourable Navdeep Bains; the Honourable Kirsty Duncan; Mr. Bill Barrable, Chief Executive Officer of the Rick Hansen Institute; and Dr. Jaimie Borisoff, Principal Investigator at ICORD

    The funding is a part of the $22.3-million Accessible Technology Program, which provides support for the development of assistive and adaptive digital devices and technologies to help Canadians with disabilities take full advantage of technology.

    The funding will allow us to continue developing an open-source assistive technology model across Canada that greatly reduces the cost of many assistive devices. Read the full press release on the Government of Canada website.

  • National AccessAbility Week

    Chad05/24/2018 at 17:40 0 comments

    May 27th – June 2nd is National AccessAbility Week, a time to promote inclusion and accessibility and to celebrate the contributions of Canadians with disabilities. We have three big events coming up with TELUS, Microsoft, and Moncton High School.

    National AccessAbility Week logo with text: Promoting accessibiltiy every day, everywhere.

    TELUS Days of Giving LipSync Buildathon: Victoria (May 28th)

    TELUS employees will volunteer their time and skills to help build LipSyncs that will go out to individuals in their communities. Want to help out or receive a LipSync? Register for the TELUS Days of Giving LipSync Buildathon.

    Moncton High School LipSync Buildathon (May 30th)

    Moncton High School students will come together to build LipSyncs for people with disabilities. Sign up if you can volunteer or if you’d like to receive a LipSync. Register for the Moncton High School LipSync Buildathon.

    Lunch and Learn with Microsoft (May 31st)

    As part of Mobility Day, Microsoft will be hosting this event with demos from Tobii Eye Tracking and us. We’ll be demonstrating the LipSync and how it can help persons with limited use of their hands. Register for the Microsoft Lunch and Learn.

    Join us to promote accessibility!

    PS: Are you a post-secondary student living with a spinal cord injury and studying in British Columbia? More than $250,000 in post-secondary funding is available this year through the Gragopean Scholarship Fund. For more information, please visit sci-bc.ca/scholarships.

View all 70 project logs

Enjoy this project?

Share

Discussions

Mark Brady Ingle wrote 07/02/2017 at 22:52 point

is anyone still working on this project?  I need some help and ordering information on the mouth piece parts

  Are you sure? yes | no

Chad wrote 07/08/2017 at 16:14 point

hey Mark glad we caight up on email..sorry not logging into hackaday as often as I should. 

  Are you sure? yes | no

Mark Brady Ingle wrote 06/27/2017 at 02:50 point

I am testing on Mac and need the instructions for the RED/Green flash sequence.  I have a bluetooth connection established but I am not able to go any further.  Help would be appreci

  Are you sure? yes | no

Trent Robertson wrote 05/30/2017 at 16:00 point

Thank you for sharing this project. One of my best friends has ALS and would benefit from this. I have already set up voice recognition features in her apartment to allow her to control lights, air conditioning, and the TV, but she still needs someone to hold and control a tablet for her if she wants to surf the internet. I hope to find the time to build this soon.

  Are you sure? yes | no

Chad wrote 07/08/2017 at 16:13 point

hey Trent let us know if we can send you some parts / the main boards!

  Are you sure? yes | no

Mark Brady Ingle wrote 05/09/2017 at 20:53 point

Hello team!  Are the current instructions online the most current?  I have enough parts to get started so I wanted to make sure. 

  Are you sure? yes | no

Chad wrote 05/11/2017 at 20:20 point

Hi Mark, we have added the most recent as of now.  Let us know if you got questions with the build and in particular ping @Ivan.Gee if you need in particular details

  Are you sure? yes | no

eroy wrote 01/02/2017 at 23:34 point

Thank you for this project! I am endlessly disappointed in the cost of assistive technology. To make something like this open source is a gesture rarely seen. 

  Are you sure? yes | no

Charles G wrote 02/26/2017 at 06:49 point

Thank you for the kind words eroy! Stay tuned in the next couple weeks as we launch a new and improved version of the LipSync at an even lower price point!

  Are you sure? yes | no

Manuel Z wrote 10/10/2016 at 19:18 point

this is an amazing project! congratulations!!!! 

  Are you sure? yes | no

Charles G wrote 10/10/2016 at 23:14 point

Thanks, Manuel!

  Are you sure? yes | no

Sadiq Mohamed wrote 09/22/2016 at 12:02 point

A very worthwhile project. I had a friend with MS who would have benefited from this. Unfortunately he died some years ago. I wish you the best of luck with your testing and look forward to the results.

  Are you sure? yes | no

Ivan wrote 09/22/2016 at 17:55 point

My condolences about your friend @Sadiq Mohamed . I am glad you believe in our project, we are working our tails off over here so we can start getting makers and users together to start building these and help increase people's quality on life. Just as a side note, our plans are to continue compiling and refining open-source assistive technologies, if you have any ideas I would love to hear them.

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates