Low Cost Open Source Eye Tracking

A system designed to be easy to build and setup. Uses two $20 webcams and open source software to provide accurate eye tracking

Public Chat
Similar projects worth following
One of my favorite pieces of technology is the digital camera. The camera is an exact analog to the human eye, and the data it produces is analogous to vision, perhaps our most important sense. I’ve always felt that there was a serious lack of great software to explore digital cameras and machine vision technology, so I decided to build my own: The Jevons Camera Viewer

One of my favorite pieces of technology is the digital camera. The camera is an exact analog to the human eye, and the data it produces is analogous to vision, perhaps our most important sense. I’ve always felt that there was a serious lack of great software to explore digital cameras and machine vision technology, so I decided to build my own: The Jevons Camera Viewer

Please see the github repository for all design files. This is a working prototype. 

Update 2018-05-06

I have tested a 0603 940nm smt led that is extremely bright to the c270 (even with the camera's ir filter still intact) I'm going to rework the camera once I receive the IR pass film and proceed from there. 

The next phase will likely be focused mostly on getting the calibration improved and getting the software to talk to the outside world. This will be done via another serial port communicating with an Arduino Bluetooth. The plan is to control some devices by simply looking at them! I may consider controlling the mouse as a quick interim project, but this has been done many times before so it's not the main focus of this project at this time. A 3d printed frame is being prototyped as well, but will take some time to get right.

More updates to come! 

Future plans:

Sorry just got some free time to do some more work on this. So the two largest todos are:

Improve calibration

The calibration is currently very bad, I want to fix that. The plan is to have a dot that moves around the screen. The user will follow the dot with their eyes. Once a few points are calibrated, the system should begin to be able to automatically reject bad points and keep good points. This should massively improve both the fatigue and the accuracy of the tracking. I would like to have the user be able to sit further from the monitor to improve comfort. There are issues with having the front camera out of alignment with the eyes, for practical reasons it is necessary, but paralax effect gets much worse the close the target is, so this should improve accuracy a lot.

Isolate eyetracking code

Right now the code base is a bit of a mess. A lot of the code is crossplatform compatible already, being based on openCV, so it should also ease the transition into making this a truely portable raspberry pi based solution. It will add to the cost dramatically but will improve usabiility and offer all sorts of opportunities for experimentation not possible with regular webcams. I tried walking around my shop on the limited 10ft leash of the USB cable and it was quite good! But I would love to be able to use this while driving or doing other cool experiments such as interviews!

Second tier todos:

Add an IR LED. 

Fortunately the camera is somewhat sensitive to IR! meaning that a much brighter IR LED can be utilized! This will allow for a much wider range of use cases.

3D printed camera holders

As easy to make as the current apparatus is, there are some serious drawbacks! The zip ties block part of the field of view and make it difficult to align the cameras properly. This can be improved with some nicely printed holders

Smoothing of Gaze Indicator

The current Gaze indicator jumps around a lot, especially when the user is blinking. By smoothing the output signal, the accuracy will improve and the indicator will be easier to monitor.

Identifying Pupil

The pupil is currently not being identified, making any vector drawing between the LED corneal reflection and the pupil impossible. Adding this may help to improve accuracy.

  • Looking into designing and manufacturing Camera PCB

    John Evans05/13/2018 at 18:54 0 comments

    2018-05-13 So after a very exhaustive search of USB capable image sensors I've pretty much given up.

    Things I've looked into to date:

    So I've been very impressed with the On-Semi MIPI CSI-2 interface sensors such as the MT9M114. My original plan was to design a tiny circuit board around on of their sensors. But after a very exhaustive search I was unable to find anything that can convert that data to motion-JPEG and put it on the USB. There is a very large and expensive chip made by cypress called the Cypress EZ-USB® CX3 which looks very cool, but is way more than this project requires.

    Additionally I looked into several 12 bit parallel interface sensors such as the AR0130CSSC00SPBA0-DP1, with the plan being to use a realtec usb 2.0 camera controller. Unfortunately I do not believe these are available to the public and I do no want to invest the time into a design with almost no support. It's difficult to find datasheets for these devices. An example being the RTS5822 if anyone has additional information about these devices, please let me know.

    I have looked into using a raspberry pi zero as an IP camera with a MIPI camera, such as the MT9M114 and a tiny pcb. Unfortunately the IP stream appears as though it may be delayed by several milliseconds. However I need to do some experimentation to confirm this. (The latency maybe extremely low in real terms, just requiring some optimization)

    What I'd really like to do is set up a raspberry pi zero as a USB camera device per this guide and stream the data from a MIPI camera through the USB:

    If anyone would like to take on this project I'd love to help out. But in the meantime I believe we're going to be stuck with the c270

View project log

Enjoy this project?



Daren Schwenke wrote 05/05/2018 at 20:12 point

You could do this without obstructing the users view by using an IR camera and and IR coated microscope slide or first surface mirror at a 45 degree angle.  I would say gut the IR filter out of a webcam but it's too small.

  Are you sure? yes | no

John Evans wrote 05/06/2018 at 01:24 point

You are correct about the ir filter I have read that it is integrated directly into the sensor, making removal Impossible. I'm working on a redesign of the glasses, The frames will be 3d printed. Your recommendation to use an IR coated mirror is excellent, thanks, I will likely continue with that. 

  Are you sure? yes | no

Tom Meehan wrote 04/24/2018 at 04:16 point

This is an amazing project, I've looked at vision/eye tracking applications for a while and yours is great.  Thanks for posting this project.

  Are you sure? yes | no

John Evans wrote 04/25/2018 at 03:22 point

Hi Tom, thank you very much. This is my first open source project so your encouragement is highly appreciated. It has been a lot of fun so far.

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates