Close
0%
0%

LipSync

An assistive tech which allows quadriplegics to use touchscreen mobile devices using a mouth-operated joystick with sip and puff controls

Similar projects worth following
Touchscreen devices have become a staple piece of technology in this day and age. Most people cannot get through the day without using their smartphone. It keeps us connected. It keeps us organized. It is an integral part of our social lives. For people in wheelchairs whom experience difficulties with fine upper body motor control, the usage of mobile devices can be very challenging. Statistics show there are over 1 million people in the United States and Canada that have very limited or no use of their hands, making touchscreen devices very difficult or impossible to use.

The "LipSync" is an assistive technology device which is being developed to allow quadriplegics and other people with limited hand use the ability to use touchscreen mobile devices by manipulation of a mouth-operated joystick with integrated sip and puff controls. We are releasing all of our work open-source, to make the Lipsync a solution that can be made at the community level for less than $300.

OVERVIEW

In 2016, upwards of 1.6 billion people in the world are using smartphones. Smartphones are becoming a staple piece of technology for many citizens in North America and around the World. Smartphones and touchscreen devices enable users to navigate around their city better, to communicate with others more freely, and to operate applications which can promote happy and productive lives. Currently, there exist a limited number of practical devices for quadriplegics to use touchscreen devices - this is where we step in.

The LipSync is an electronic device which allows quadriplegics the ability to use compatible touchscreen and computer devices without the use of their hands. The user is able to manipulate a cursor on their device screen using a mouth-operated joystick with integrated sip and puff controls to simulate the actions of "tap" and hitting the back button, respectively. With longer sips and longer puffs, additional secondary features are enabled including a "tap and drag", "long tap and drag" and the possibility of more specialized functions as per the user's needs.

The LipSync is design specifically for portable devices, it does not require AC power, but it will work with any device including desktop and laptop computers that support mice through a universal serial bus (USB) or the Bluetooth connection.

The LipSync is an open-source hardware project where all of our 3D printer files, component lists and microcontroller code are made public. In the spirit of accessibility, our housing can be 3D printed, the electronic components are readily available and the assembly is as straightforward as possible.

The LipSync was envisioned as a holistic solution that takes in to consideration not only the interface but how the system is to be mounted on the user’s wheelchair. The actual electronics of the device is on part of the implementation. There are no standardized methods to which wheelchairs are designed. Wheelchair manufacturers can use round or square tubing. They often also use tubing which is not compatible with other manufacturers so that customers must buy accessories from them. As a result, there is not a standard location or clamping mechanism to mount assistive technology on the chair.

Wheelchairs are also customized to the user, including the height, width and seating position of the user. The seating on the wheelchairs are customized to minimize the incidents of pressure sores. As a result, the mounting system for assistive technology such as the LipSync also has to be customized. In addition to the instructions to assemble a LipSync, we have included instructions on how to mount a LipSync. A combination of off-theshelf and custom 3D printed components are provided in order to help makers create a fully integrated and customized solution for the user.

Now the maker community and disability community can meet, collaborate and work together constructing a LipSync over a period not much longer than a weekend. We hope these new relationships will continue spurring innovation within the maker community.

There are 3 main aspects we will be addressing in our project:

  1. Developing an easy to build, but robust electronics assembly that novice to experienced makers can build;
  2. Developing a device housing which can be 3D printed by makers either at home or at dedicated facilities;
  3. Creating mounting options for a variety of wheelchairs with 3D printed parts and/or commercially available components.

SMARTPHONES AND ACCESSIBILITY

Smartphones, by their very nature, are intended to be used while on the go. Traditional assistive technology designs for the desktop and laptop computer are not portable in nature, so they cannot be easily applied to smartphones. While single and dual input systems exist for smartphones, they are slow and frustrating to use for users with more capabilities of movement. Single and dual input switches are appropriate for users that can only make one or two movements consistently. The LipSync is...

Read more »

  • National AccessAbility Week

    Chad3 days ago 0 comments

    May 27th – June 2nd is National AccessAbility Week, a time to promote inclusion and accessibility and to celebrate the contributions of Canadians with disabilities. We have three big events coming up with TELUS, Microsoft, and Moncton High School.

    National AccessAbility Week logo with text: Promoting accessibiltiy every day, everywhere.

    TELUS Days of Giving LipSync Buildathon: Victoria (May 28th)

    TELUS employees will volunteer their time and skills to help build LipSyncs that will go out to individuals in their communities. Want to help out or receive a LipSync? Register for the TELUS Days of Giving LipSync Buildathon.

    Moncton High School LipSync Buildathon (May 30th)

    Moncton High School students will come together to build LipSyncs for people with disabilities. Sign up if you can volunteer or if you’d like to receive a LipSync. Register for the Moncton High School LipSync Buildathon.

    Lunch and Learn with Microsoft (May 31st)

    As part of Mobility Day, Microsoft will be hosting this event with demos from Tobii Eye Tracking and us. We’ll be demonstrating the LipSync and how it can help persons with limited use of their hands. Register for the Microsoft Lunch and Learn.

    Join us to promote accessibility!

    PS: Are you a post-secondary student living with a spinal cord injury and studying in British Columbia? More than $250,000 in post-secondary funding is available this year through the Gragopean Scholarship Fund. For more information, please visit sci-bc.ca/scholarships.

  • Golden State Week

    Chad05/17/2018 at 16:25 0 comments

    Harry, our Manager of Research and Development, and Jake, our Mechanical Engineer, are coming down to sunny California for two events.

    Chad pointing at a Maker Faire sign

    San Francisco Maker Faire (May 18th - 20th)

    It's the biggest Maker Faire on Earth (or as they say, the Greatest Show and Tell)! Harry and Jake will be at our booth all three days, so stop by and say hi if you're around.

    On the 20th, Harry will sit on a panel called Making Access for People with Disabilities. He'll talk about Makers Making Change alongside four brilliant makers who will cover adaptations for persons with different disabilities.

    Sign up for the San Francisco Maker Faire here.

    Demonstration of keyguard usage

    Laser Cutting Keyguards Workshop (May 22nd)

    We're making the most of our stay in the Bay Area and hosting this workshop in partnership with the Communication Technology Education Center (CTEC) and HackerLab.

    Keyguards can be very useful for people with disabilities, helping them control their input and address mobility and agility issues. We're sending our best laser cutter, Jake, to lead this workshop.

    Sign up for the Laser Cutting Keyguard workshop here.

    We hope to see you soon!

  • Help Us Print Low Tech AT Kits

    Chad05/10/2018 at 21:24 0 comments

    Makers Making Change has started reaching out to Occupational Therapists (OTs). OTs work directly with people with disabilities and can connect them with our services, giving people access to more affordable, community-made assistive devices.

    Help Us Print Low Tech AT Kits

    Completed LipSyncs at the TELUS Vancouver Buildathon in 2017
    Our Low Tech AT Kits are composed of simple 3D-printed devices from our Open Source Assistive Technology Library. We will share these kits with OTs, whose clients will be able to try out the devices. 

    This will help us get valuable feedback on how well these devices function and how they can be improved to meet the needs of people with disabilities.

    Contents of a Makers Making Change Low Tech AT Kit

    If you want to get involved, please get in touch with Zee, the Makers Making Change Project Manager, by emailing her at zeek@neilsquire.ca. She will connect you with a hospital or disability organization in your area where you can drop off the kit.

    Thank you for your support!

  • TELUS Days of Giving LipSync Buildathons

    Chad05/03/2018 at 21:50 0 comments

    Makers Making Change have partnered with TELUS to host six TELUS Days of Giving LipSync Buildathons across Canada. We invite you to join us at any of these events, especially if you or someone you know in these communities could benefit from a LipSync.

    Upcoming TELUS Days of Giving LipSync Buildathons

    Upcoming TELUS Days of Giving LipSync Buildathons

    The first one is coming up tomorrow, May 4th, in Calgary. Register here for the buildathon in Calgary.

    When: 8:00 AM – 4:00 PM MDT

    Where2912 Memorial Dr SE, Calgary, AB T2A 6R1

    Our deep appreciation and thanks to TELUS for providing the funding, staff as volunteers, and support in helping us build over 100 LipSyncs through their Days of Giving to benefit people with disabilities.

  • Buildathons and More Buildathons

    Chad04/26/2018 at 17:39 0 comments

    We at Makers Making Change are happy to announce some upcoming buildathons. We have an exciting (and exhausting) list of events planned for May, June, and July. 

    Completed LipSyncs at the TELUS Vancouver Buildathon in 2017

    Here’s a sneak peek at the ones in May:

    May 4: Buildathon in Calgary
    May 12: Buildathon in the Okanagan
    May 14: BC Tech Summit LipSync Buildathon
    May 22: Laser-cut keyguard workshop in California
    May 25: Buildathon in Vancouver
    May 28: Buildathon in Victoria

    Save these dates! We'll share more details over the upcoming weeks. We are looking forward to collaborating with makers all over North America as we build devices that help people with disabilities.

    PS: This Saturday, April 28th, we will be in Philadelphia for the NextFab Accessibility Hackathon during Philly Tech Week 2018. Pop by and say hi if you’re in the area.

  • A Visit from the Honourable Kirsty Duncan

    Chad04/19/2018 at 18:39 0 comments

    Every week is science week at Makers Making Change. We really hopped on the science train this past week, though.

    A Visit from the Minister

    On April 12th, we were thrilled to welcome the Honourable Kirsty Duncan, Minister of Science and Minister of Sport and Persons with Disabilities, to the Neil Squire Society’s Burnaby Head Office. We showcased the LipSync and other low-tech assistive devices such as a pen holder and nail clipper holder.

    Jim, a quadriplegic, demonstrates the LipSync to Minister Duncan

    Jim demonstrates a LipSync to Minister Duncan

    Chad Leaman, Director of Innovation, shows the Minister a pen holder that slips on the palm

    Chad shows the Minister a palm pen holder

    View more photos from the Minister's visit here.

    STAN Conference

    On April 11th, Chad was the keynote speaker at the STAN 2018 conference here in Vancouver. Held at Science World, the event was all about building bridges and creating communities in STEAM. That's a big part of our work at Makers Making Change!

    Chad Leaman delivering the keynote at STAN 2018 with a Makers Making Change backdrop

    Chad delivering the keynote at STAN 2018

    Science World Community Scientist Initiative

    We were back at Science World on April 15th when Jake, our Mechanical Engineer, hosted a booth as part of the Community Scientist Initiative. The initiative gives all Science World visitors an opportunity to meet and learn from a science-based professional.

    Jake's booth at Science World

    Jake showing children some asistive writing devices like the palm pen holder and pen ball

    Phew – that was a busy few days. Now, we are down in Utah for the AOTA Conference.

  • 2018 AOTA Annual Conference

    Chad04/12/2018 at 18:39 0 comments

    Our Makers Making Change team will be making a beeline for Salt Lake City, Utah, to attend the American Occupational Therapy Association (AOTA) Annual Conference April 19–22. There, we'll be connecting with occupational therapists and local resource organizations to spread the message of open-source technologies benefitting people with disabilities.

    2018 AOTA Annual Conference

    We're excited to add that we'll be hosting a pre-conference workshop on Wednesday, April 18th: The Occupational Therapist’s Maker Toolbox: DIY Assistive Technology Solutions in the Clinic. Attendees will learn about 3D-printing, soldering, and making different assistive technology solutions.

    Last year was our first time at the AOTA Conference and we discovered there was huge interest in the concept of making. Some occupational therapists who had been in the industry for over 30 to 40 years spoke about their experiences with tinkering in order to create their own assistive solutions.

    We're looking forward to connecting with even more occupational therapists and local resource groups this year, furthering our goal of connecting people with disabilities to makers.

  • Meet Fawzan, a 16-Year-Old Maker

    Chad04/05/2018 at 16:47 0 comments

    Grade 10 student Fawzan has founded Fraser Heights Youth for CARE, a student group that raises awareness of Surrey Memorial Hospital while engaging youth in the community.

    Fawzan and his group 3D-printed pen balls, palm pen holders, bottle openers, and other devices from our open-source Assistive Technology Library to help people with disabilities.

    Fawzan with an EEG headset

    Above: Fawzan in our Makers Making Change lab, demonstrating the MindWave EEG (electroencephalogram) headset. He is using the device for a science fair experiment, measuring links between attention span and math scores

    AT devices on a table

    Above: Some pen holders and in white, a bottle opener that the Youth for CARE group created

    Fawzan says: "I strive to follow Mahatma Gandhi's advice: be the change you wish to see in this world. I wish to use my knowledge and skills to help and make a positive change in my community. I also like to encourage my peers to assist others in need."

    We're glad youth like Fawzan are coming together to do some good in their communities. We hope to continue hearing such amazing stories throughout the year!

  • We Built and Had Fun on World Create Day

    Chad04/05/2018 at 16:30 0 comments

    On World Create Day, we worked on some (very cool) projects that would empower people with disabilities. In Vancouver, we created custom video game controllers, prototyped exo-skeleton arms, and repaired LipSyncs.

    We were joined by students and volunteers. Here’s the team in action:

    Top view of Mimi working on an accessible Wii controller

    Above: Mimi working on an accessible Wii controller

    Albert and Wen-Ling

    Above: UBC capstone team members, (left) Albert and (right) Wen-Ling, showing off their work on an updated LipSync running on Adafruit Feather board

    SFU capstone team working on exo-skeleton arm

    Above: SFU capstone team working their way through an exo-skeleton arm (and snacks – snacks are important)

    Thank you for helping us build something that matters.

  • Remembering Stephen Hawking

    Chad03/22/2018 at 19:00 0 comments

    We are sad to hear of the passing of Stephen Hawking. Along with being one of the most brilliant minds of his time, he was also a steadfast advocate of assistive technology. He insisted that the technology which helped him communicate be open-source, in order to be accessible to everyone who needed it.

    Stephen Hawking at NASA in 2008

    In honour of Professor Hawking, CBC's The Current aired a segment on assistive technology. Dr. Gary Birch, Executive Director of the Neil Squire Society, was fortunate to be a guest on the show. You can listen to the segment here.

    PS: The system Intel developed for Professor Hawking is publicly available as the Assistive Context-Aware Toolkit (ACAT). You can explore the ACAT files on GitHub. Open-source technology is a big part of our work at Makers Making Change, so we believe this is useful and important.

View all 61 project logs

Enjoy this project?

Share

Discussions

Mark Brady Ingle wrote 07/02/2017 at 22:52 point

is anyone still working on this project?  I need some help and ordering information on the mouth piece parts

  Are you sure? yes | no

Chad wrote 07/08/2017 at 16:14 point

hey Mark glad we caight up on email..sorry not logging into hackaday as often as I should. 

  Are you sure? yes | no

Mark Brady Ingle wrote 06/27/2017 at 02:50 point

I am testing on Mac and need the instructions for the RED/Green flash sequence.  I have a bluetooth connection established but I am not able to go any further.  Help would be appreci

  Are you sure? yes | no

Trent Robertson wrote 05/30/2017 at 16:00 point

Thank you for sharing this project. One of my best friends has ALS and would benefit from this. I have already set up voice recognition features in her apartment to allow her to control lights, air conditioning, and the TV, but she still needs someone to hold and control a tablet for her if she wants to surf the internet. I hope to find the time to build this soon.

  Are you sure? yes | no

Chad wrote 07/08/2017 at 16:13 point

hey Trent let us know if we can send you some parts / the main boards!

  Are you sure? yes | no

Mark Brady Ingle wrote 05/09/2017 at 20:53 point

Hello team!  Are the current instructions online the most current?  I have enough parts to get started so I wanted to make sure. 

  Are you sure? yes | no

Chad wrote 05/11/2017 at 20:20 point

Hi Mark, we have added the most recent as of now.  Let us know if you got questions with the build and in particular ping @Ivan.Gee if you need in particular details

  Are you sure? yes | no

eroy wrote 01/02/2017 at 23:34 point

Thank you for this project! I am endlessly disappointed in the cost of assistive technology. To make something like this open source is a gesture rarely seen. 

  Are you sure? yes | no

Charles G wrote 02/26/2017 at 06:49 point

Thank you for the kind words eroy! Stay tuned in the next couple weeks as we launch a new and improved version of the LipSync at an even lower price point!

  Are you sure? yes | no

Manuel Z wrote 10/10/2016 at 19:18 point

this is an amazing project! congratulations!!!! 

  Are you sure? yes | no

Charles G wrote 10/10/2016 at 23:14 point

Thanks, Manuel!

  Are you sure? yes | no

Sadiq Mohamed wrote 09/22/2016 at 12:02 point

A very worthwhile project. I had a friend with MS who would have benefited from this. Unfortunately he died some years ago. I wish you the best of luck with your testing and look forward to the results.

  Are you sure? yes | no

Ivan wrote 09/22/2016 at 17:55 point

My condolences about your friend @Sadiq Mohamed . I am glad you believe in our project, we are working our tails off over here so we can start getting makers and users together to start building these and help increase people's quality on life. Just as a side note, our plans are to continue compiling and refining open-source assistive technologies, if you have any ideas I would love to hear them.

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates