Recently, in one of our work pitches at Firstborn, we were asked whether we could create a web-based application that could use your webcam to track the position of your eyes. We didn’t know for sure that would be possible (or to which level of accuracy) so we decided to spend a few days creating a test prototype. This is the result, running on Flash 10.

The red circle represents the perceived position of my eyes, and the green line is just a line that moves with a speed that is based on the circle position (when the circle is on top, the line moves up; when the circle is in the middle, the line stays; when the circle is at the bottom, the line moves down).

It uses a face tracking algorithm for initial face detection, then some other color-separating code to find where the eye is looking at in real-time. An initial calibration is necessary (the code must know when the eye is looking at the top and at the bottom of the screen; then it can find where you’re looking at, in between those two).

The face detection used an AS3 port of OpenCV, based on the work of Ohtsuka Masakazu and Mario Klingemann with a bunch of modifications for speed and varying levels of accuracy (nothing too crazy though, just better for our specific use case).

The result was SoBe’s Staring Contest, where you have to keep your eyes in a specific part of the screen to “win” the game (try the “experimental version” for the eye-tracking webcam version; otherwise it’s a mouse-based game). It doesn’t work in all situations – it relies heavily on the amount of light available at the visitor’s environment, and on the visitor’s ability to not rotate their head too much during the experience – but I still consider it a success.

As a developer, it’s very important to take some time out to try not only new techniques but also new platforms. The tools we have available for us change constantly and to keep up you need to make time for some private investigation of new methods and capabilities. This time for research & development with something that may or may not be fruitful is hard to come by, but it’s certainly of utmost importance in this career. It’s also fun, of course.

This recent presentation by Joon Park, our Chief Creative Officer, talks a little about this, talking not only about the above example but also about some other great “personal” projects by some of our developers that started as an exercise but turned out to be so much more.

P.S. As always, we’re hiring.