Dmai, Inc.

Ashwin Dharne

Nishant Shukla

The present teaching relates to method, system, medium, and implementations for managing a user machine dialogue. A request is received by a server from a device for a response to be directed to a user engaged in a dialogue with the device. The request includes information related to a current state of the dialogue. The response is determined based on a dialogue tree and the information related to the current state of the dialogue. A sub-dialogue tree, which corresponds to a portion of the dialogue tree, is then created based on the response and the dialogue tree and is then used to generate a local dialogue manager for the device. The response, the sub-dialogue tree, and the local dialogue manager are then sent to the device, wherein the local dialogue manager, once deployed on the device, is capable of driving the dialogue with the user based on the sub-dialogue tree on the device.

System and method for conversational agent via adaptive caching of dialogue tree

The present teaching relates to method, system, medium, and implementations for managing a user machine dialogue. Sensor data is received at a device, including an utterance representing a speech of a user engaged in a dialogue with the device. The speech of the user is determined based on the utterance and a response to the user is searched by a local dialogue manager residing on the device against a sub-dialogue tree stored on the device. The response, if identified from the sub-dialogue tree, is rendered to the user in response to the speech. A request is sent to a server for the response, if the response is not available in the sub-dialogue tree.

The present teaching relates to method, system, medium, and implementations for detecting a source of speech sound in a dialogue. A visual signal acquired from a dialogue scene is first received, where the visual signal captures a person present in the dialogue scene. A human lip associated with the person is detected from the visual signal and tracked to detect whether lip movement is observed. If lip movement is detected, a first candidate source of sound is generated corresponding to an area in the dialogue scene where the lip movement occurred.

System and method for disambiguating a source of sound based on detected lip movement

The present teaching relates to method, system, medium, and implementations for speech recognition. An audio signal is received that represents a speech of a user engaged in a dialogue. A visual signal is received that captures the user uttering the speech. A first speech recognition result is obtained by performing audio based speech recognition based on the audio signal. Based on the visual signal, lip movement of the user is detected and a second speech recognition result is obtained by performing lip reading based speech recognition. The first and the second speech recognition results are then integrated to generate an integrated speech recognition result.

Growing community of inventors

Irvine, CA, United States of America

Ashwin Dharne