At&t Intellectual Property I, L.p.

Nuance Communications, Inc.

At+t Corp.

Patrick John Ehlen

Michael J Johnston

Brant Jameson Vasilieff

David Crawford Gibbon

Behzad Shahraray

Zhu Liu

Mazin E Gilbert

Jay Henry Lieske, Jr

Helen Hastie

Jay Gordon Wilpon

Ethan Selfridge

Hyuckchul Jung

Remi Zajac

Disclosed herein are systems, methods, and non-transitory computer-readable storage media for processing audio. A system configured to practice the method monitors, via a processor of a computing device, an image feed of a user interacting with the computing device and identifies an audio start event in the image feed based on face detection of the user looking at the computing device or a specific region of the computing device. The image feed can be a video stream. The audio start event can be based on a head size, orientation or distance from the computing device, eye position or direction, device orientation, mouth movement, and/or other user features. Then the system initiates processing of a received audio signal based on the audio start event. The system can also identify an audio end event in the image feed and end processing of the received audio signal based on the end event.

System and method for enhancing speech activity detection using facial feature detection

Methods, systems, devices, and media for creating a plan through multimodal search inputs are provided. A first search request comprises a first input received via a first input mode and a second input received via a different second input mode. The second input identifies a geographic area. First search results are displayed based on the first search request and corresponding to the geographic area. Each of the first search results is associated with a geographic location. A selection of one of the first search results is received and added to a plan. A second search request is received after the selection, and second search results are displayed in response to the second search request. The second search results are based on the second search request and correspond to the geographic location of the selected one of the first search results.

System and method for creating and sharing plans through multimodal dialog

A system, method and computer-readable storage devices are disclosed for multi-modal interactions with a system via a long-touch gesture on a touch-sensitive display. A system operating per this disclosure can receive a multi-modal input comprising speech and a touch on a display, wherein the speech comprises a pronoun. When the touch on the display has a duration longer than a threshold duration, the system can identify an object within a threshold distance of the touch, associate the object with the pronoun in the speech, to yield an association, and perform an action based on the speech and the association.

System and method for initiating multi-modal speech recognition using a long-touch gesture

The invention provides for a system, method, and computer readable medium storing instructions related to controlling a presentation in a multimodal system. The method embodiment of the invention is a method for the retrieval of information on the basis of its content for real-time incorporation into an electronic presentation. The method comprises receiving from a presenter a content-based request for at least one segment of a first plurality of segments within a media presentation and while displaying the media presentation to an audience, displaying to the presenter a second plurality of segments in response to the content-based request. The computing device practicing the method receives a selection from the presenter of a segment from the second plurality of segments and displays to the audience the selected segment.

System and method for controlling presentations using a multimodal interface

The invention provides for a system, method, and computer readable medium storing instructions related to controlling a presentation in a multimodal system. The method embodiment of the invention is a method for the retrieval of information on the basis of its content for incorporation into an electronic presentation. The method comprises receiving from a user a content-based request for at least one segment from a first plurality of segments within a media presentation preprocessed to enable natural language content searchability; in response to the request, presenting a subset of the first plurality of segments to the user; receiving a selection indication from the user associated with at least one segment of the subset of the first plurality of segments and adding the selected at least one segment to a deck for use in a presentation.

System and method for creating a presentation using natural language

Methods, systems, devices, and media for creating a plan through multimodal search inputs are provided. A multimodal virtual assistant receives a first search request which comprises a geographic area. First search results are displayed in response to the first search request being received. The first search results are based on the first search request and correspond to the geographic area. Each of the first search results is associated with a geographic location. The multimodal virtual assistant receives a selection of one of the first search results, and adds the selected one of the first search results to a plan. A second search request is received after the selection, and second search results are displayed in response to the second search request being received. The second search results are based on the second search request and correspond to the geographic location of the selected one of the first search results.

There is provided for a system, method, and computer readable medium storing instructions related to controlling a presentation in a multimodal system. A method for the retrieval of information on the basis of its content for real-time incorporation into an electronic presentation is discussed. One method includes controlling a media presentation using a multimodal interface. The method involves receiving from a presenter a content-based request associated with a plurality of segments within a media presentation preprocessed for context-based searching; displaying the media presentation and displaying to the presenter results in response to the content-based request; receiving a selection from the presenter of at least one result; and displaying the selected result to an audience.

System and method for using speech for data searching during presentations

Computational models of dialog context have often focused on unimodal spoken dialog or text, using the language itself as the primary locus of contextual information. But as spoken unimodal interaction is replaced by situated multimodal interaction on mobile platforms supporting a combination of spoken dialog with graphical interaction, touch-screen input, geolocation, and other non-linguistic contextual factors, a need arises for more sophisticated models of context that capture the influence of these factors on semantic interpretation and dialog flow. The systems, methods, and computer program products disclosed herein address this need. A method for multimodal search includes, in part, determining an intended location of search query based upon information received from a remote mobile device that issued the search query.

Systems, methods, and computer program products for location salience modeling for multimodal search

In one or more embodiments, one or more methods and/or systems described can perform producing a lattice of object hypotheses based on multiple reference objects from image information; receiving input speech information that includes a request for information associated with at least one reference object of the multiple reference objects; producing a lattice of speech hypotheses based on at least a first possible description included in the speech information; producing a lattice of scored semantic hypotheses based on at least the lattice of object hypotheses and the lattice of speech hypotheses; determining that a single semantic interpretation score of the lattice of scored semantic hypotheses exceeds a predetermined value; and providing requested information associated with the at least the first reference object of the plurality of reference objects.

Growing community of inventors

San Francisco, CA, United States of America

Patrick John Ehlen