Qcue, Inc.

Avi Barliya

Yonatan Wexler

Aviad Maizels

Systems and methods are disclosed for determining an intent to speak based on minute facial skin movements. In one implementation, a system may include a processor configured to control at least one coherent light source to illuminate a region of a face. The processor may receive from at least one sensor, reflection signals indicative of coherent light reflected from the face. The reflection signals may be analyzed to determine minute facial skin movements associated with silent speech. Then, based on the determined minute facial skin movements associated with the silent speech, the processor may determine a speech intent, and initiate, prior to an audible utterance of the silent speech, an action based on the determined speech intent.

Initiating an action based on a detected intention to speak

Systems and methods are disclosed for speech detection from facial skin movements. In one implementation, a system may include at least one light source, at least one sensor configured to receive light reflections from the at least one light source; and a processor configured to control the at least one light source to illuminate a region of a face of a user in a time interval. The processor may receive reflection signals indicative of light reflected from the face in the time interval. The reflection signals may be analyzed to determine facial skin movements in the time interval, wherein determining the sequence of words includes using an artificial neural network and a motion-to-language analysis. Then, based on the determined facial skin movements, the processor may determine a sequence of words associated with the facial skin movements, and output a textual transcription corresponding with the determined sequence of words.

Speech detection from facial skin movements

Head mountable systems, methods, and non-transitory computer readable media including instructions for identifying individuals using facial skin micromovements are disclosed. An example head mountable system may include a wearable housing, a coherent light source, a detector, and at least one processor. The at least one processor may analyze reflection signals from the detector to determine specific facial skin micromovements of an individual wearing the head mountable system. Thereafter, the at least one processor may access memory correlating a plurality of facial skin micromovements with the individual, and search for a match between the determined specific facial skin micromovements and at least one of the plurality of facial skin micromovements in the memory. If a match is identified, the processor may initiate a first action; and if a match is not identified, the at least one processor may initiate a second action different from the first action.

Using facial skin micromovements to identify a user

Systems, methods, and non-transitory computer readable media including instructions for interpreting facial skin micromovements are disclosed. An example head mountable system includes a housing configured to be worn on a head of a wearer. The head mountable system also includes at least one detector integrated with the housing and configured to receive light reflections from a facial region of the head and to output associated reflection signals. The head mountable system also includes at least one microphone associated with the housing and configured to capture sounds produced by the wearer and to output associated audio signals. The head mountable system also includes and at least one processor in the housing, configured to use both the reflection signals and the audio signals to generate output that corresponds with words articulated by the wearer.

Using a wearable to interpret facial skin micromovements

Systems, methods, and non-transitory computer readable media including instructions for noise suppression are described. A head mountable system for noise suppression includes a wearable housing; a coherent light source configured to project light towards a facial region of the head; a detector configured to receive coherent light reflections from the facial region associated with facial skin micromovements and to output associated reflection signals; and a processor configured to: analyze the reflection signals to determine speech timing; receive audio signals from at least one microphone; correlate the reflection signals with the received audio signals to determine portions of the audio signals associated with the words spoken by the wearer; and output the determined portions of the audio signals associated with words spoken by the wearer, while omitting output of other portions of the audio signals not containing the words spoken by the wearer.

Wearable for suppressing sound other than a wearer's voice

Systems, methods, and non-transitory computer readable media including instructions for establishing nonvocalized conversations are disclosed. The operations include establishing a wireless communication channel for enabling a nonvocalized conversation via a first wearable device and a second wearable device. The operations also include detecting by the first wearable device first facial skin micromovements occurring in an absence of perceptible vocalization, and transmitting a first communication from the first wearable device to the second wearable device, wherein the first communication is derived from the first facial skin micromovements and is transmitted for presentation via the second wearable device. The operations also include receiving a second communication from the second wearable device, wherein the second communication is derived from second facial skin micromovements detected by the second wearable device, and presenting the second communication to a wearer of the first wearable device.

Facilitating silent conversation

Systems, methods, and computer program products are disclosed for removing noise from facial skin micromovement signals. Removing noise from facial skin micromovements includes, during a time period when an individual is involved in at least one non-speech-related physical activity, operating a light source in a manner enabling illumination of a facial skin region of the individual; receiving signals representing light reflections from the facial skin region; analyzing the received signals to identify a first reflection component indicative of prevocalization facial skin micromovements and a second reflection component associated with the at least one non-speech-related physical activity; and filtering out the second reflection component to enable interpretation of words from the first reflection component indicative of the prevocalization facial skin micromovements.

Interpreting words prior to vocalization

Systems, methods, and non-transitory computer readable media including instructions for performing operations for continuous authentication based on facial skin micromovements is disclosed. The operations may include receiving during an ongoing electronic transaction, first signals representing coherent light reflections associated with first facial skin micromovements during a first time period and second signals representing coherent light reflections associated with second facial skin micromovements during a second time period. The operations may also include determining, using the first and second signals, that a specific individual is associated with the first and second facial skin micromovements. The operations may also include receiving during the ongoing electronic transaction third signals representing coherent light reflections associated with third facial skin micromovements. The operations may further include determining, using the third signals, that the third facial skin micromovements are not associated with the specific individual, and initiating an action based on the determination.

Using pattern analysis to provide continuous authentication

Systems, methods, and computer program products are disclosed for initiating content interpretation operations prior to vocalization of content to be interpreted. Initiating content interpretation operations prior to vocalization of content to be interpreted includes receiving signals representing facial skin micromovements; determining from the signals at least one word to be spoken prior to vocalization of the at least one word in an origin language; prior to the vocalization of the at least one word, instituting an interpretation of the at least one word; and causing the interpretation of the at least one word to be presented as the at least one word is spoken.

Systems and methods are disclosed for determining textual transcription from minute facial skin movements. In one implementation, a system may include at least one coherent light source, at least one sensor configured to receive light reflections from the at least one coherent light source; and a processor configured to control the at least one coherent light source to illuminate a region of a face of a user. The processor may receive from the at least one sensor, reflection signals indicative of coherent light reflected from the face in a time interval. The reflection signals may be analyzed to determine minute facial skin movements in the time interval. Then, based on the determined minute facial skin movements in the time interval, the processor may determine a sequence of words associated with the minute facial skin movements, and output a textual transcription corresponding with the determined sequence of words.

Speech transcription from facial skin movements

Wearable coherent light sensing systems and methods are disclosed. In one implementation, a system may include a wearable coherent light source configured to direct light towards a facial area; a wearable sensor configured to receive light source reflections and to output associated reflection signals; and a processor operable in an idle mode and in a high power mode. In the idle mode, the processor is configured to receive the reflection signals, process the reflection signals to identify trigger in the reflection signals, and automatically switch to the high power mode upon identification of the trigger. In the high power mode, the processor is configured to analyze the reflection signals to identify facial movements associated with silent speech. Following the identification of the facial movements associated with the silent speech, the processor is configured to decipher the facial movements and generate an output associated with the silent speech.

Facial movements wake up wearable

A method for generating speech includes uploading a reference set of features that were extracted from sensed movements of one or more target regions of skin on faces of one or more reference human subjects in response to words articulated by the subjects and without contacting the one or more target regions. A test set of features is extracted a from the sensed movements of at least one of the target regions of skin on a face of a test subject in response to words articulated silently by the test subject and without contacting the one or more target regions. The extracted test set of features is compared to the reference set of features, and, based on the comparison, a speech output is generated, that includes the articulated words of the test subject.

Growing community of inventors

Tel Aviv, Israel

Avi Barliya