International Business Machines Corporation

Nuance Communications, Inc.

Chalapathy Venkata Neti

Gerasimos Potamianos

Sankar Basu

Stephane Herman Maes

Andrew William Senior

Nitendra Rajput

Ashish Verma

Satyanarayana Venkata Nitta

Mukesh Kumar Mohania

Prasenjit Dey

L Venkata Subramaniam

Bikram Sengupta

Malolan Chetlur

Ganesh Nachiappa Ramaswamy

Robert J Schloss

Sabine V Deligne

Giridharan R Iyengar

Philippe Christian De Cuetos

Jonathan Hudson Connell, Ii

John Richard Smith

Ravindranath Kokku

Padmanabha Venkatagiri Seshadri

Ching-Yung Lin

Jan Kleindienst

Belle L Tseng

Norman C Haas

Etienne Marcheret

Danish Contractor

Upendra V Chaudhari

Sumit Negi

Tanveer Afzal Faruquie

Salim Estephan Roukos

Daniel Mark Coffman

Geoffrey G Zweig

Aditya Sinha

Homayoon S M Beigi

Hugh William Adams, Jr

Popani Gopalakrishnan

Benoit Emmanuel Ghislain Maison

Philippe De Cuetos

Tanveer Atzal Faruquie

Giridharen Iyengar

Harriet J Nock

A computer-implemented method for a socially connectable agent. A non-limiting example of the computer-implemented method includes receiving, by a processor, input events. The method includes evaluating and enacting rules, by the processor, based on the received input events, where the rule include stochastic elements and temporal operators. The method pushes, by the processor, action events that result from the evaluation and enactment of the rules, and initiates, by the processor, action events.

Rules engine for social learning

Providing targeted, evidence-based recommendations to improve digital content quality, in one aspect, may include partitioning digital content into a sequence of logical content modules. Quality scores for the logical content modules along a plurality of quality dimensions may be determined. User interactions with the digital content on a user interface are monitored. Raw events generated from user interaction in accessing the digital content are captured. Raw event patterns for the logical content modules are detected. The raw event patterns of user interaction are transformed into a set of dynamic states representing potential user experience and performance. The set of dynamic states are associated with a respective logical content module. The quality dimensions of the logical content modules and associated quality scores are correlated with the dynamic states.

Providing targeted, evidence-based recommendations to improve content by combining static analysis and usage analysis

Methods and arrangements for identifying burden comprehension in multimedia content. A contemplated method includes: accepting multimedia input; detecting components of the multimedia input; determining a comprehension burden score of each of the detected components; and thereupon calculating a total comprehension burden score for the multimedia input. Other variants and embodiments are broadly contemplated herein.

Identification of comprehension burden in multimedia content

A manual annotation system of multi-modal characteristics in multimedia files. There is provided an arrangement for selection an observation modality of video with audio, video without audio, audio with video, or audio without video, to be used to annotate multimedia content. While annotating video or audio features is isolation results in less confidence in the identification of features, observing both audio and video simultaneously and annotating that observation results in a higher confidence level.

System and method for annotating multi-modal characteristics in multimedia documents

An arrangement for yielding enhanced audio features towards the provision of enhanced audio-visual features for speech recognition. Input is provided in the form of noisy audio-visual features and noisy audio features related to the noisy audio-visual features.

Audio-visual codebook dependent cepstral normalization

Bootstrapping of a system from one language to another often works well when the two languages share the similar acoustic space. However, when the new language has sounds that do not occur in the language from which the bootstrapping is to be done, bootstrapping does not produce good initial models and the new language data is not properly aligned to these models. The present invention provides techniques to generate context dependent labeling of the new language data using the recognition system of another language. Then, this labeled data is used to generate models for the new language phones.

Language context dependent data labeling

Techniques for performing audio-visual speech recognition, with improved recognition performance, in a degraded visual environment. For example, in one aspect of the invention, a technique for use in accordance with an audio-visual speech recognition system for improving a recognition performance thereof includes the steps/operations of: (i) selecting between an acoustic-only data model and an acoustic-visual data model based on a condition associated with a visual environment; and (ii) decoding at least a portion of an input spoken utterance using the selected data model. Advantageously, during periods of degraded visual conditions, the audio-visual speech recognition system is able to decode (recognize) input speech data using audio-only data, thus avoiding recognition inaccuracies that may result from performing speech recognition based on acoustic-visual data models and degraded visual data.

Audio-only backoff in audio-visual speech recognition system

Automated decision making techniques are provided. For example, a technique for generating a decision associated with an individual or an entity includes the following steps. First, two or more data streams associated with the individual or the entity are captured. Then, at least one time-varying measure is computed in accordance with the two or more data streams. Lastly, a decision is computed based on the at least one time-varying measure. One form of the time-varying measure may include a measure of the coverage of a model associated with previously-obtained training data by at least a portion of the captured data. Another form of the time-varying measure may include a measure of the stability of at least a portion of the captured data. While either measure may be employed alone to compute a decision, preferably both the coverage and stability measures are employed. The technique may be used to authenticate a speaker.

Automated decision making using time-varying stream reliability prediction

The use of multiple complementary classes of measure to assess face and speech consistency in video. In an exemplary embodiment, both synchrony measures and plausibility measures are employed.

Assessing consistency between facial motion and speech signals in video

Systems and methods are provided for performing focus detection, referential ambiguity resolution and mood classification in accordance with multi-modal input data, in varying operating conditions, in order to provide an effective conversational computing environment for one or more users.

System and method for multi-modal focus detection, referential ambiguity resolution and mood classification using multi-modal input

Techniques for providing speech recognition comprise the steps of processing a video signal associated with an arbitrary content video source, processing an audio signal associated with the video signal, and recognizing at least a portion of the processed audio signal, using at least a portion of the processed video signal, to generate an output signal representative of the audio signal.

Method and apparatus for audio-visual speech detection and recognition

A computer implemented method in a language independent system generates audio-driven facial animation given the speech recognition system for just one language. The method is based on the recognition that once alignment is generated, the mapping and the animation hardly have any language dependency in them. Translingual visual speech synthesis can be achieved if the first step of alignment generation can be made speech independent. Given a speech recognition system for a base language, the method synthesizes video with speech of any novel language as the input.

Translingual visual speech synthesis

A system for activating a microphone based on visual speech cues, in accordance with the invention, includes a feature tracker coupled to an image acquisition device. The feature tracker tracks features in an image of a user. A region of interest extractor is coupled to the feature tracker. The region of interest extractor extracts a region of interest from the image of the user. A visual speech activity detector is coupled to the region of interest extractor and measures changes in the region of interest to determine if a visual speech cue has been generated by the user. A microphone is turned on by the visual speech activity detector when a visual speech cue has been determined by the visual speech activity detector. Methods for activating a microphone based on visual speech cues are also included.

System and method for microphone activation using visual speech cues

Growing community of inventors

Yorktown Heights, NY, United States of America

Chalapathy Venkata Neti