Massachusetts Institute of Technology

Paul Duchnowski

Louis D Braida

Maroula S Bratakos

David S Lum

Matthew Sexton

Jean S Krause

In a computer-based method for use in speech cueing, speech elements initially delivered by a speaker are recognized. A sequence of video images is displayed showing the speaker delivering the speech elements. The displayed video sequence is delayed relative to the initial delivery of the speech elements by the speaker. In conjunction with displaying the sequence of video images, images of cues corresponding to the recognized speech elements are displayed with a timing that is synchronized with the delayed sequence of video images. Each of the cues comprises a representation of a human hand in a discrete position. Successive different cues are displayed in a manner to suggest to a viewer smooth motion between the successive cues. The speech elements are recognized based on an acoustic signal from the speaker as the speech elements are delivered. Images of cues corresponding to the recognized speech elements are displayed with a selected timing that controls the synchronization of the display of the visual cues relative to timing of the speech elements. Cues can be displayed with a supplementary visual feature to enhance comprehension by a cue receiver. Additionally, various speech recognition techniques are provided.

Automatic cueing of speech

A method for recognizing speech elements (e.g., phones) in utterances includes the following steps. Based on acoustic frequency, at least two different acoustic representatives are isolated for each of the utterances. From each acoustic representative, tentative decision information on the speech element in the corresponding utterance is derived. A final decision on the speech element in the utterance is then generated, based on the tentative decision information derived from more than one of the acoustic representatives.

Growing community of inventors

Brookline, MA, United States of America

Paul Duchnowski