Wireless real-time tongue tracking for speech impairment diagnosis, speech therapy with audiovisual biofeedback, and silent speech interfaces

Abstract

A real-time wireless system for recording natural tongue movements in the 3D oral space. By attaching a small magnetic tracer to the tongue, either temporarily or semi-permanently, and placing an array of magnetic sensors around the mouth, the tracer can be localized with sub-millimeter precision. The system can also combine the tracer localization with acoustic, video, and flow data via additional sensors to form a comprehensive audiovisual biofeedback mechanism for diagnosing speech impairments and improving speech therapy. Additionally, the system can record tongue trajectories and create an indexed library of such traces. The indexed library can be used as a tongue tracking silent speech interface. The library can synthesize words, phrases, or execute commands tied to the individual patterns of magnetic field variations or tongue trajectories.