Microsoft Technology Licensing, LLC

Michael Levit

Shuangyu Chang

Sarangarajan Parthasarathy

Bruce Melvin Buntschuh

Benoit F Dumoulin

Issac Alphonso

Andreas Stolcke

Marcus Jon Jager

Ziad Al Bawab

Padma Varadharajan

Karthik Raghunathan

Anand U Desai

Umut Ozertem

Nick Kibre

Rishi Girish

Ananth Rampura Sheshagiri Rao

Hernan Guelman

Tao Li

Abhik Lahiri

Barlas Oguz

Dhiresh Rawal

Suyash Choudhury

Xin Meng

Cem Aksoylar

Anastasios Anastasakos

Nitin Agarwal

Systems and methods for speech recognition correction include receiving a voice recognition input from an individual user and using a trained error correction model to add a new alternative result to a results list based on the received voice input processed by a voice recognition system. The error correction model is trained using contextual information corresponding to the individual user. The contextual information comprises a plurality of historical user correction logs, a plurality of personal class definitions, and an application context. A re-ranker re-ranks the results list with the new alternative result and a top result from the re-ranked results list is output.

Error correction in speech recognition

Systems and methods are provided for acquiring training data and building an organizational-based language model based on the training data. In organizational data is generated via one or more applications associated with an organization, the collected organizational data is aggregated and filtered into training data that is used for training an organizational-based language model for speech processing based on the training data.

Organizational-based language model generation

Provided is a system and method for acquiring training data and building an organizational-based language model based on the training data. In one example, the method may include collecting organizational data that is generated via one or more applications associated with an organization, aggregating the collected organizational data with previously collected organizational data to generate aggregated organizational training data, training an organizational-based language model for speech processing based on the aggregated organizational training data, and storing the trained organizational-based language model.

The customization of language modeling components for speech recognition is provided. A list of language modeling components may be made available by a computing device. A hint may then be sent to a recognition service provider for combining the multiple language modeling components from the list. The hint may be based on a number of different domains. A customized combination of the language modeling components based on the hint may then be received from the recognition service provider.

Flexible schema for language model customization

A computer system for language modeling may collect training data from one or more information sources, generate a spoken corpus containing text of transcribed speech, and generate a typed corpus containing typed text. The computer system may derive feature vectors from the spoken corpus, analyze the typed corpus to determine feature vectors representing items of typed text, and generate an unspeakable corpus by filtering the typed corpus to remove each item of typed text represented by a feature vector that is within a similarity threshold of a feature vector derived from the spoken corpus. The computer system may derive feature vectors from the unspeakable corpus and train a classifier to perform discriminative data selection for language modeling based on the feature vectors derived from the spoken corpus and the feature vectors derived from the unspeakable corpus.

Language modeling based on spoken and unspeakable corpuses

Systems and methods are provided for optimizing language models for in-domain applications through an iterative, joint-modeling approach that expresses training material as alternative representations of higher-level tokens, such as named entities and carrier phrases. From a first language model, an in-domain training corpus may be represented as a set of alternative parses of tokens. Statistical information determined from these parsed representations may be used to produce a second (or updated) language model, which is further optimized for the domain. The second language model may be used to determine another alternative parsed representation of the corpus for a next iteration, and the statistical information determined from this representation may be used to produce a third (or further updated) language model. Through each iteration, a language model may be determined that is further optimized for the domain.

Language model optimization for in-domain application

An incremental speech recognition system. The incremental speech recognition system incrementally decodes a spoken utterance using an additional utterance decoder only when the additional utterance decoder is likely to add significant benefit to the combined result. The available utterance decoders are ordered in a series based on accuracy, performance, diversity, and other factors. A recognition management engine coordinates decoding of the spoken utterance by the series of utterance decoders, combines the decoded utterances, and determines whether additional processing is likely to significantly improve the recognition result. If so, the recognition management engine engages the next utterance decoder and the cycle continues. If the accuracy cannot be significantly improved, the result is accepted and decoding stops. Accordingly, a decoded utterance with accuracy approaching the maximum for the series is obtained without decoding the spoken utterance using all utterance decoders in the series, thereby minimizing resource usage.

Incremental utterance decoder combination for efficient and accurate decoding

Optimized language models are provided for in-domain applications through an iterative, joint-modeling approach that interpolates a language model (LM) from a number of component LMs according to interpolation weights optimized for a target domain. The component LMs may include class-based LMs, and the interpolation may be context-specific or context-independent. Through iterative processes, the component LMs may be interpolated and used to express training material as alternative representations or parses of tokens. Posterior probabilities may be determined for these parses and used for determining new (or updated) interpolation weights for the LM components, such that a combination or interpolation of component LMs is further optimized for the domain. The component LMs may be merged, according to the optimized weights, into a single, combined LM, for deployment in an application scenario.

Token-level interpolation for class-based language models

Various components provide options to re-format an input based on one or more contexts. The input is received that has been submitted to an application (e.g., messaging application, mobile application, word-processing application, web browser, search tool, etc.), and one or more outputs are identified that are possibilities to be provided as options for re-formatting. A respective score of each output is determined by applying a statistical model to a respective combination of the input and each output, the respective score comprising a plurality of context scores that quantify a plurality of contexts of the respective combination. Exemplary contexts include historical-user contexts, domain contexts, and general contexts. One or more suggested outputs are selected from among the one or more outputs based on the respective scores and are provided as options to re-format the input.

Context-aware re-formating of an input

On a computing device a speech utterance is received from a user. The speech utterance is a section of a speech dialog that includes a plurality of speech utterances. One or more features from the speech utterance are identified. Each identified feature from the speech utterance is a specific characteristic of the speech utterance. One or more features from the speech dialog are identified. Each identified feature from the speech dialog is associated with one or more events in the speech dialog. The one or more events occur prior to the speech utterance. One or more identified features from the speech utterance and one or more identified features from the speech dialog are used to calculate a confidence score for the speech utterance.

Leveraging interaction context to improve recognition confidence scores

Examples of the present disclosure describe generation of a multi-arc confusion network to improve, for example, an ability to return alternatives to output generated. A confusion network comprising token representations of lexicalized hypotheses and normalized hypotheses is generated. Each arc of the confusion network represents a token of a lexicalized hypothesis or a normalized hypothesis. The confusion network is transformed into a multi-arc confusion network, wherein the transforming comprising realigning at least one token of the confusion network to span multiple arcs of the confusion network. Other examples are also described.

Growing community of inventors

San Jose, CA, United States of America

Michael Levit