Conv-eye-ing: An eye-tracking software to empower people with ALS
Anirudha Ghosh and Anirban Ghosh
Conv-eye-ing is an application that helps people with ALS communicate with ease. It uses computer vision to identify and track the human pupil and convert those movements into text. This was inspired by the mode of communication used by the acclaimed guitarist Jason Becker, developed by his father Gary Becker.
The system works by first identifying faces presented to the camera. Once that is done, the system identifies the eyes and then locates and tracks the pupil.
Once that is done, the person moves their pupil in the direction of one of the 6 cells. Once the cell is identified, the person moves their eye to the desired letter. In essence, every letter of the alphabet is a combination of two eye movements.
On the right, you can see the initial prototype built on python using python and openCV.
For the second prototype, openCV and dlib were used which improved accuracy and pupil capture.
Figma prototype for the app: