Nanjing University

College of William and Mary

Yafeng Yin

Lei Xie

Qun Li

Sanglu Lu

Systems, methods and techniques are provided for interacting with mobile devices using a camera-based keyboard. The system comprises a processor system including at last one processor. The processor system is configured to at least capture a plurality of images in connection with the keyboard and at least one hand typing on the keyboard via the camera. Based on the plurality of captured images, the processor system is further configured to locate the keyboard, extract at least a portion of the keys on the keyboard, extract a hand, and detect a fingertip of the extracted hand. After that, a keystroke may be detected and localized through tracking the detected fingertip in at least one of the plurality of captured images, and a character corresponding to the localized keystroke may be determined.

Growing community of inventors

Nanjing, China

Yafeng Yin