Close
0%
0%

Eye Controlled Wheelchair!

Open source system to drive powerchairs by eye movement alone - allowing independent mobility when use of a person's hands isn't an option.

Similar projects worth following
People who have totally lost the use of their muscles cannot operate their own wheelchairs.

The Eyedrivomatic system takes advantage of existing eye tracking technology to allow users to drive their chairs again - using only their eyes. It's a low cost, open source way to give mobility back to people who thought they had lost it forever.

The Problem

Spinal injury is the most common cause of quadriplegia, though there are many diseases that can produce the same result - inability to produce controlled movement in any of your limbs or head. For people in this situation independent control of their wheelchairs is not possible – the technology just isn't available. Independent mobility increases quality of life significantly, and it's loss is keenly felt by those robbed of it, through accident or disease.

The scale of the problem is enormous. ALS is responsible for only a tiny fraction of the people in this situation, yet there are five thousand ALS sufferers at any one time in the UK alone, and twenty thousand in the US. Most of them will end up sat immobile in their wheelchairs, unable to move on their own. Eyedrivomatic could help most of them.

The Concept

To create an inexpensive and accessible way for people with motorised wheelchairs to take control of their mobility via eye movement, using open source methods as much as possible.

Our team

Patrick Joyce - Team captain

Steve Evans - Test pilot

David Hopkinson - Film maker

Cody Barnes - Software Developer

Steve and I both have Motor Neuron Disease, or ALS as it's also known. We both have electric wheelchairs, but while I still have some remaining movement in my fingers, and am just able to operate mine, Steve only has his eyes left, and until Eyedrivomatic, had to rely on his carer to operate his.

Background

It is usual for people with MND/ALS in the UK to be provided with electric wheelchairs loaned by the health service, for as long as we need them. Steve and I have wheelchairs, and we both also have eyegaze equipment (which is becoming increasingly more available). At the moment it is not possible to operate one with the other. Eyegaze technology is only intended for operating a computer, not the chair.

The Original Idea

As I don't actually own either the wheelchair or the eyegaze, my idea was to make something that would interface with the user's chair mounted computer, and physically move the joystick. Crucially this would mean not making any modifications to the loaned hardware, and could work with any wheelchair and eyegaze combination.

The solution I came up with is called Eyedrivomatic, and is a two part system, a 'brain box', containing an arduino and four relays, and an 'electronic hand', containing servos to move the wheelchair's joystick.

I decided on this approach, rather than interfacing directly with the wheelchair's electronics for several reasons. I know the protocols for my own chair, but not all chairs use the same protocol. Also, my chair has a socket for plugging in external joysticks, but that is by no means common on other chairs. And the sockets are of a proprietary design. Interfacing directly would, on most chairs, involve physically cutting in to the wheelchair's wiring.

This is an issue because many people don’t actually own their chairs – especially in the UK where they are loaned by the health service and will be reclaimed after the user’s death.

An integrated version of eyedrivomatic in new chairs would be brilliant, but this hacking – yet non-interfering – approach to existing tech means the problem can be solved cheaply and easily for those who need it as soon as possible.

This is especially important as ALS usually has a very rapid onset and people losing their mobility need a practical and inexpensive solution as soon as possible with what is immediately available.

Two years in development, eyedrivomatic now works well, and is;

  • Easy to use
  • Safe
  • Expandable - the system is able to control other external devices according to the needs of the user
  • Totally open, both open hardware and open source software
  • Capable of being built at home - by an unskilled person
  • Soon to be a manufactured product as well

I put in a four relay module as standard equipment in the brain...

Read more »

  • 1 × 3d printed parts. The case doesn't have to be 3d printed, but the electronic hand does.
  • 1 × Arduino uno R3 £6 - £16 Original Arduino or a compatible one.
  • 1 × 4 channel relay shield (5v) £7 a specific model is required. See build instructions for photo
  • 1 × servo/sensor shield £5 a specific model is required. See build instructions for photo
  • 1 × 5v - 7v power supply for the servo's (see update) £19 Optional. 5v power supply for the servos. Possible to use your wheelchair's battery
  • 2 × Turnigy 380 max micro servo £15 or similar high torque micro servos
  • 1 × usb A to usb B cable - to go between pc and brain box £1.50
  • 1 × USB A to DC plug cable £1.50 the particular type of DC plug isn't important, as it will be removed
  • 3 × 3.5mm sockets with attached cable (mono) £2 each. Buy mono extension cables and cut the ends off
  • 2 × Servo extension cables £2 - £4 These come in a variety of lengths. 30cm is fine if you mount your brain box near the joystick.

View all 19 components

  • More sad news from the Eyedrivomatic team

    Cody Barnes3 days ago 7 comments

    The hacker/maker community has just suffered another great loss. I was informed this morning that Patrick Joyce has also passed away. Although I know Patrick from his work on the Eyedrivomatic, he was involved in several projects, several of them listed here on Hackaday.io, that so clearly demonstrated his compassion, creativity, intelligence and inability to give-up. I am very grateful to Patrick and to Steve for allowing me to work on this project with them and intend to continue with their work. There are many people out there suffering with MND or other conditions that have left them unable to move. And thanks to Patrick's vision, those people have gained a small part of their independence back. This sounds cliche, but it is true none the less... Patrick and Steve did their part to make the world a better place.

  • Sad News from the Eyedrivomatic Team

    Cody Barnes4 days ago 13 comments

    I am sorry to report that after more than a decade of fighting Motor Neurone Disease (MND), on April 14th, Steve Evans has passed away. MND is a brutal terminal disease that slowly destroys the ability of its victims to function and removes almost all self-reliance while leaving the mind fully in tact. Despite his own troubles, Steve selflessly found a way through Eyedrivomatic to help many others with similar struggles. Up to the end, Steve was busy communicating with users and researchers and coordinating manufacturing and delivery. I cannot understate how much Steve was a key part of the Eyedrivomatic team, and the loss of Steve is a major blow to the project.

    There will be a short delay in the production of any new units while we figure out how to pick-up where Steve left off. However despite our loss, the Eyedrivomatic team intends to continue development and has much planned for this year.

    If you are interested in helping with this project, please send me a note. We have much still to do and without Steve, we could use some help! If you could benefit from Eyedrivomatic, I would also like to hear from you. If you can't participate and have no personal use for Eyedrivomatic, but would like to share your appreciation of Steve's work, I would like to hear from you too (and would be happy to forward your notes to Steve's family).

    Sincerely,

    Cody Barnes

  • Power Problem

    Patrick Joyce04/22/2016 at 09:21 0 comments

    BREAKING NEWS

    Good news and bad news, bad first. The 'powerbee executive' power pack component, that provides power to the servos, is no longer available. Most other usb powerbanks have internal circuitry that turns them off when power demand drops. Unfortunately they don't switch back on when power demand resumes. Eyedrivomatic, inconveniently, only requires power sporadically. We are currently looking for a solution, and will post updates here soon.
    Good news now. The all singing, all dancing, stand alone pc application that we're writing, is nearly finished! (many thanks to our genius coder – Cody Barnes).
    More news soon....

  • ​ Eyedrivomatic News!!

    Patrick Joyce03/03/2016 at 11:56 0 comments

    I've been very lazy since the end of last years Hackaday Prize competition. However, over the last month i've pulled myself together, and got back on the horse, so to speak. By the end of the competition we'd put everything up on github, thingiverse and instructables. All the files necessary to build your own eyedrivomatic system. But it wasn't as good as we wanted. The hardware was ok, but the software had two issues. It was tricky to set up, and only worked in conjunction with a commercial software package called Grid2.

    The tricky setup issue was due to the com port number that a pc assigns to an arduino (like the one in the brain box ). The processing app for eyedrivomatic needs to know that number. Each user had to discover his own com port number, and manually alter the code for the app. Simple for a geek. Hard indeed for a normal person – daunting even.
    That problem has now disappeared, thanks to our brand new Eyedrivomatic Port Finder app. Written by W. A. Smith, and modified by Steve Thomas, port finder finds out your com port number, and stores it in a config file on your pc – ready for the main eyedrivomatic app to use.
    Its brilliantly simple. You run port finder once only, and then the main eyedrivomatic app runs perfectly. Simple.
    The new port finder app is up our Github repository, along with the modified main eyedrivomatic app.
    Its just the Grid2 issue to deal with now. Give me a few more weeks, and eyedrivomatic will work on any pc, with or without Grid2, simply!

  • Beta testing begins!

    Patrick Joyce10/24/2015 at 14:15 2 comments

    Since the project's inception, the waiting list of quadriplegics wanting their own system has been growing steadily. So it was with great pleasure that i posted the first three beta systems out on Monday. One of them is headed for Jim Durfer, a US desert storm veteran with ALS. His system is yet to arrive. Jono Stenburg, a twenty four year old film maker with cerebral palsy, and Steve Thomas, an ex computer programmer, also with ALS, live here in the UK. This is a video of their first go......

  • Frantic! Progress! Frantic!

    Patrick Joyce10/21/2015 at 10:57 0 comments

    Frantic? That word doesn't begin to describe what it's been like round here lately. Despite the huge workload of preparing for the finals, in an act of insanity, three weeks ago, I decided to completely redesign the brain box. One of the parts in the old brain box had become hard to get hold of. This prompted the redesign, but I ended up redesigning all of it. Dupont cables are out, along with the LED's. The system now uses a 4 channel relay shield on top of the arduino uno, with a servo/sensor shield on top of that. This simplifies the build process enormously. Everything plugs in, or gets screwed in to a terminal block.

    I also took the opportunity to add another safety feature. Previously, when the eyedrivomatic app was started, the servos would occasionally give a random jerk. Which was potentially dangerous. So in the new brain box I used one of the relays to deny power to the servos, until both the app, and the firmware were running smoothly.

    The electronic hand has also had two revisions in the last few weeks. It is now at Mk6! I've also got some exciting news about our three beta testers. But i'll put that in the next update!

  • Eyedrivomatic indoors!

    Patrick Joyce10/05/2015 at 14:22 0 comments

    Most of our previous videos have been filmed outside. Partly for the light, and partly, with some of the early prototypes, because they were not quite accurate enough for narrow interior spaces. No longer. With the new Mk5 electronic hand, and updated software, Eyedrivomatic is now precise, accurate and reliable. To demonstrate this, here is Steve at home, driving about inside.

  • ALS and the Mk5 hand

    Patrick Joyce09/19/2015 at 09:57 0 comments

    Myself and Steve both have ALS. When you've had it for a while, like we have, your health becomes very fragile. Steve had pneumonia recently, which prevented him from working on Eyedrivomatic for a month. Last week was my turn. I can no longer swallow, and for the last two years I have been fed through a tube in my stomach. Last week I stopped being able to tolerate the food at all. After three days without food I was rushed into hospital. The doctors fiddled with my medication, and the situation improved a little. I am out of hospital now, and feeding again - enough to keep me alive, though with a distressing amount of nausea attached. This makes perfecting Eyedrivomatic all the more urgent. While it works well now, there are too many loose ends for others to take over easily were I to die unexpectedly. The system works well, but it needs more fine tuning on the electronic hand, and time spent making the software more plug and play.

    The Mk5 electronic hand is now in it's fifth sub prototype. The left right servo was moving excessively compared to the forwards backwards one, so I redesigned it. I moved the pivot point for the main cradle one centimetre lower, which hopefully will restore the balance between the x and y movement. I'm posting it to Steve for testing today. Meanwhile i'm redesigning it a further time as it's too hard to build at the moment. An easy build process is very important to us, as the end user is unlikely to be an engineer.

    We've had a lot of interest from disabled people wanting their own Eyedrivomatic systems. As soon as the Mk5b has proved itself, we'll start producing units for beta testing.

  • Eyedrivomatic at the Race Track

    Patrick Joyce09/17/2015 at 17:50 0 comments

    Just a quick video update to show Steve in his natural environment.

  • Mk5 Electronic Hand

    Patrick Joyce08/27/2015 at 08:09 2 comments

    I printed my design for the Mk5 electronic hand, unfortunately the design wasn't quite as perfect as i'd hoped. There have been two sub prototypes since then to improve clearances and performance. The third one (the one with red servo arms), is good enough to use for preliminary testing, so i'll be posting it to Steve today.

    The Mk5 is much harder to assemble than the Mk4, so i'm already working on an easier to build version.

    I'm really keen to expand our testing team – to start getting eyedrivomatic out to the people that need it, so i've started the process of recruiting beta testers. As soon as the new electronic hand proves itself in the field, i'll produce four more systems and get them out to the new crew....

View all 18 project logs

View all instructions

Enjoy this project?

Share

Discussions

Thane Pullan wrote 12/25/2015 at 04:23 point
hi, i have cerebral palsy and am developing eye tracking and communication software, please contact me via my contact form. i would like to help you. thanepullan.com

  Are you sure? yes | no

Thane Pullan wrote 12/25/2015 at 04:23 point
hi, i have cerebral palsy and am developing eye tracking and communication software, please contact me via my contact form. i would like to help you. thanepullan.com

  Are you sure? yes | no

kstindt wrote 12/12/2015 at 23:24 point

I am writing an on-line module for OCALI ATIM on AT for mobility and would love to include your story, pix and videos. Please contact me about permission kstindt@cesa6.org.

and keep up the amazing work!!!

  Are you sure? yes | no

lawrence wrote 11/18/2015 at 20:16 point

Hi, I am based in Taunton and interested in production once design is ready. Ping me an email and I am happy to drop by for a chat.

  Are you sure? yes | no

matt venn wrote 11/18/2015 at 16:41 point

well done to you all, I was moved by your videos. If you still need help with a PCB, let me know.

  Are you sure? yes | no

John Blance wrote 11/18/2015 at 06:14 point

Amazing work and a worthy winner

I'm impressed on the potential benefit to people's lives with this project.

Is there a way to get involved or help? 

Keep up the great work and I hope you can help many with this!!

John

  Are you sure? yes | no

Patrick Joyce wrote 11/18/2015 at 12:49 point

thanks for the offer John! You can get involved and help. Yes please. Things are manic hereat themoment, i'll message you in a week or so, when it's calmed down a bit 

  Are you sure? yes | no

Craig Hissett wrote 11/17/2015 at 02:01 point

Well done! Worthy winner!

  Are you sure? yes | no

Aleksandar Bradic wrote 11/17/2015 at 01:27 point

Congrats!!

  Are you sure? yes | no

The Big One wrote 11/16/2015 at 15:40 point

Congrats, guys!  Great project and well deserved win.

  Are you sure? yes | no

MW Motors wrote 11/15/2015 at 07:37 point

Hi Patrick,

Congratulations on winning the HaD prize. Keep up the great work.  Best regards. Maurice

  Are you sure? yes | no

Patrick Joyce wrote 11/18/2015 at 12:42 point

thanks maurice! 
I was amazed that you didn't place though....

  Are you sure? yes | no

MW Motors wrote 11/18/2015 at 12:58 point

Hi Patrick,

I won the car !!..

Well done.. Best regards. Maurice

  Are you sure? yes | no

Sebastian Nielsen wrote 11/15/2015 at 05:09 point

What about a photoresistor, that detects the level of light, and turns on/turns off the safety function automatically? So immediately when the users drive outside in strong sunlight, the safety function is activated, and when user drives inside and light level falls to a safe level to reliable detect the users eye, safety function turns off. Of course this could be configurable. Like Safety=Off, Safety=On, Safety=Auto.

And just curious, isn't is possible to configure eyegaze to move the mouse position to a safe position (like stop button) once the Contact with user's Eyes is lost?

  Are you sure? yes | no

Patrick Joyce wrote 11/18/2015 at 12:38 point

thanks for theidea! The problem is the the way light affects eyegaze equipment is quite complex. Its not simply more or less light. Eyegaze uses ir light to detect contrast. A change in the ambient ir level can throw off the calibration, so moving at all causes problems. Also it is not a problem if the eyegaze equipment loses focus because it takesthe mouse away. The real problem is whenit loses focus, but doesn't realise it 

  Are you sure? yes | no

Bob Paradiso wrote 06/23/2015 at 23:10 point

Dear Patrick,

The work you guys have done is great! 

I've worked on a similar project: http://bobparadiso.com/2015/06/16/eye-gaze-controlled-wheelchair-robotic-arm-and-ecu/

But, indeed, one very very useful thing about your approach is that it 'should' just work with any joystick. You don't need to do any further hacking to work on a new joystick. Mine you have to work out each new joystick protocol.

I'm very interested in how your project progresses, and very excited to see that instructable!

Thanks,

-Bob

  Are you sure? yes | no

Patrick Joyce wrote 06/24/2015 at 09:07 point

Bob, you're a genius! I love it. I am particularly excited about the robot arm. It was in my mind to make an arm for my wheelchair . However, your use of the actuator as the main arm is simply genius. I would never have thought of that. I even have one in my project bits box. Thank you lots! 

  Are you sure? yes | no

waqarshahzad wrote 06/14/2015 at 12:20 point

how  much time did ittake to complete?

  Are you sure? yes | no

Patrick Joyce wrote 06/15/2015 at 08:01 point

If you mean, how long does it take to build an eyedrivomatic system yourself? Once you have all the components and 3d printed parts, maybe an afternoon. If you mean how long have we spent on the project - two years.

  Are you sure? yes | no

waqarshahzad wrote 06/14/2015 at 12:17 point

okk bro i come here daily n waiting....  must give precautions to use it thankuu

  Are you sure? yes | no

Daniel wrote 06/11/2015 at 14:46 point

Hi Patrick, I did a multi-input system for people with cerebral palsy and others, to control wheelchairs, robotics arms and computers some months ago. You can see some of the videos (sry, only in Spanish) here:  and 
Remember to be careful with mechanical devices moving the wheelchair's joystick, be sure to include a safety system to cut the power, just in case, and that for some wheelchairs the 9g servos are not enough.

Good luck and congratulations for your project, it looks wonderful! :)

  Are you sure? yes | no

Patrick Joyce wrote 06/11/2015 at 17:38 point

Thats really cool. I love the robot arm. And thanks for the advice. We are very aware of the safety issues around eyedrivomatic. The safety thing has always come first in the design process. The 9g servos that we are using are high torque ones, and the next version of the hand will be able to take standard and micro servos depending on your joystick weight. Myself and Steve both have lightened joysticks, so for us smaller was more beautiful... 

  Are you sure? yes | no

waqarshahzad wrote 06/09/2015 at 05:23 point

Hyy Patrick....

your work is fantasticc....me doing same project,... 

can u help mee... thankuuu

  Are you sure? yes | no

Patrick Joyce wrote 06/09/2015 at 07:13 point

I'm happy to help if I can. What help do you need? 

  Are you sure? yes | no

waqarshahzad wrote 06/11/2015 at 04:22 point

can u send me details .. method,& components,,, plz

want it 4 my mother

  Are you sure? yes | no

Patrick Joyce wrote 06/11/2015 at 12:55 point

We are not quite ready to put the how to instructions up yet, because the project is still in active and rapid development. Also, because Eyedrivomatic is potentially dangerous, we want to make sure its as safe as it can possibly be, before we let others use it. But don't worry. It won't be long - within a month, and certainly before August, I expect to have all the instructions on here for anyone to use.

  Are you sure? yes | no

Carolina Palacios wrote 05/13/2015 at 11:48 point

Dear Patrick, congratulations on this project. You are an inspiration to many and I really hope you guys win the trip to space -  you so deserve it. Best of luck and keep up the amazing work.  

  Are you sure? yes | no

Patrick Joyce wrote 05/13/2015 at 15:26 point

thanks!  

  Are you sure? yes | no

Sophi Kravitz wrote 05/01/2015 at 18:37 point

This project is excellent. Like very much and look forward to seeing progress!

  Are you sure? yes | no

Patrick Joyce wrote 05/02/2015 at 09:11 point

Thank you, Sophi! 

  Are you sure? yes | no

malvasio.christophe wrote 04/30/2015 at 22:14 point

you should power your wheelchair with RPI2 instead

that's what i'll do for me ;)

  Are you sure? yes | no

Patrick Joyce wrote 05/02/2015 at 09:11 point

The reason for using arduino for this project is that the servos in the electronic hand only requirea simple microprocessor. Arduino is cheap and simple. But the main reason is that I am an electronics beginner, and arduino is all I know. 

  Are you sure? yes | no

malvasio.christophe wrote 05/02/2015 at 12:39 point

RPI2 is enough to enable anti-collision; auto-cruise ...

and temporary replace the mounted computer if needed

for few buck more

we spend our life in wheelchair  so it have to be the best ;)

  Are you sure? yes | no

Jon Campbell wrote 04/27/2015 at 13:59 point

There has been some similar work done at Washington State University and Tulane University with regard to the over-the-joystick configuration. Both projects were coordinated via Team Gleason, so checking with Eric Valor should be able to get you more details. http://blog.jaybeavers.org/3rd-generation-wheelchair-interface/ has some good info on how to hook up an Arduino to an R-Net Omni or similar device. While the R-Net is an add-on device, it does provide a consistent and well tested interface point since it is designed to work across multiple chair platforms and is the standard way for providing switch based mobility control.

  Are you sure? yes | no

Patrick Joyce wrote 04/29/2015 at 14:09 point

Wow! Thanks Jon. Thats amazing! I have an omni on my own chair, and had
certainly considered interfacing with it directly. However I assumed
(wrongly) that the technology would be proprietary and unavailable to
someone like me. Thank you - I will pursue this further.

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates