One of my favorite pieces of technology is the digital camera. The camera is an exact analog to the human eye, and the data it produces is analogous to vision, perhaps our most important sense. I’ve always felt that there was a serious lack of great software to explore digital cameras and machine vision technology, so I decided to build my own: The Jevons Camera Viewer
Please see the github repository for all design files. This is a working prototype.
I have tested a 0603 940nm smt led that is extremely bright to the c270 (even with the camera's ir filter still intact) I'm going to rework the camera once I receive the IR pass film and proceed from there.
The next phase will likely be focused mostly on getting the calibration improved and getting the software to talk to the outside world. This will be done via another serial port communicating with an Arduino Bluetooth. The plan is to control some devices by simply looking at them! I may consider controlling the mouse as a quick interim project, but this has been done many times before so it's not the main focus of this project at this time. A 3d printed frame is being prototyped as well, but will take some time to get right.
More updates to come!
Sorry just got some free time to do some more work on this. So the two largest todos are:
The calibration is currently very bad, I want to fix that. The plan is to have a dot that moves around the screen. The user will follow the dot with their eyes. Once a few points are calibrated, the system should begin to be able to automatically reject bad points and keep good points. This should massively improve both the fatigue and the accuracy of the tracking. I would like to have the user be able to sit further from the monitor to improve comfort. There are issues with having the front camera out of alignment with the eyes, for practical reasons it is necessary, but paralax effect gets much worse the close the target is, so this should improve accuracy a lot.
Isolate eyetracking code
Right now the code base is a bit of a mess. A lot of the code is crossplatform compatible already, being based on openCV, so it should also ease the transition into making this a truely portable raspberry pi based solution. It will add to the cost dramatically but will improve usabiility and offer all sorts of opportunities for experimentation not possible with regular webcams. I tried walking around my shop on the limited 10ft leash of the USB cable and it was quite good! But I would love to be able to use this while driving or doing other cool experiments such as interviews!
Second tier todos:
Add an IR LED.
Fortunately the camera is somewhat sensitive to IR! meaning that a much brighter IR LED can be utilized! This will allow for a much wider range of use cases.
3D printed camera holders
As easy to make as the current apparatus is, there are some serious drawbacks! The zip ties block part of the field of view and make it difficult to align the cameras properly. This can be improved with some nicely printed holders
Smoothing of Gaze Indicator
The current Gaze indicator jumps around a lot, especially when the user is blinking. By smoothing the output signal, the accuracy will improve and the indicator will be easier to monitor.
The pupil is currently not being identified, making any vector drawing between the LED corneal reflection and the pupil impossible. Adding this may help to improve accuracy.