Close

It's about more than just tapping the screen: Features of the LipSync

A project log for LipSync

An assistive tech which allows quadriplegics to use touchscreen mobile devices using a mouth-operated joystick with sip and puff controls

makers-making-changeMakers Making Change 05/17/2017 at 16:240 Comments

We’ve talked a lot about the LipSync being a sip and puff system, so for example, you puff on the tube and it taps the screen. But you do a lot more with your phone than just tap the screen — sometimes you swipe the screen left and right, sometimes you’re scrolling up and down, sometimes you just need to hit the back button.

With the LipSync, we’ve built in a few different functions to mimic those hand gestures with a sip and puff device.

Omar testing out LipSync

Omar trying the latest iteration of the LipSync.

For example, on your mobile device, if you need to tap and drag (say you need to input a pattern into the security lock screen, or simply scroll a small distance down a page), you do a medium-length puff while moving the mouthpiece cursor. If you need to do a quick scroll (instead of having to move the cursor up and down constantly to go down, you can just use this feature to skip down 2/3rds of the screen), you can do a medium-length sip, for about three seconds.

Hitting the back button is as easy as a quick sip. If you begin to experience cursor drift, you can resolve the issue by doing a long puff (about five seconds) to do a home position reset.

If you were to use a LipSync on a desktop computer, many of the functions are quite similar, but instead accommodate a more mouse-based platform (as opposed to a smartphone that’s designed to be navigated with your fingers). A medium-length puff will click and drag, while a short sip will operate as a right click.

And keep in mind that these features will be refined in further user testing. As with everything we’ve done, some of these features will be as easy to use and intuitive as we intended when we designed them, others may need to be altered.

It has to be accessible in the manner that there’s not a steep learning curve — that might turn off first-time users. It’s got to feel natural after a little bit of use, and that’s what we’re refining.

PS To read more on the functions of a LipSync and how to get it started, read our LipSync Start-Up Guide. Much of the info we’ve talked about here is on page 8.

Discussions