Microsoft Technology Licensing, LLC

Michael D Plumpe

Xuedong D Huang

Julian James Odell

Jonathan E Hamaker

Jon Hamaker

Rob Chambers

Christopher Dac Le

Alejandro Acero

Christian Liensberger

Varsha Mahadevan

Onur Domanic

Qiang Wu

Li Deng

David Mowatt

Philipp Heinz Schmid

Michael V Calcagno

Robert I Brown

Keith Coleman Herold

Travis Wilson

James L Adcock

Rachel Imogen Morton

Robert Edward Dewar

Ricky Dee Loynd

Sandeep Manocha

Michael Calcagno

Techniques are described herein that are capable of implementing a context-aware digital personal assistant (DPA) that supports multiple accounts and/or facilitating interaction among digital personal assistants. For example, a user may be signed-in with accounts of a DPA. Content from content streams associated with the respective accounts may be selectively combined based on at least the user's context. In another example, users who are signed-in with accounts of a DPA may share a user experience provided by the DPA. Content from content streams associated with the respective accounts may be selectively combined based on at least one or more of the users' context. In yet another example, a first DPA associated with a first user may be caused to perform an operation on behalf of a second DPA associated with a second user or to delegate the operation to the second DPA.

Facilitating interaction among digital personal assistants

Context-aware digital personal assistant supporting multiple accounts

The present disclosure describes an exemplary hybrid client/server architecture that may be utilized leverage the unique capabilities of both remote and local services. Data may be processed in parallel by remote and local processes. Results generated during the parallel processing may be exchanged between remote and local services and used to update results generated by the separate services. The hybrid client/server architecture may be utilized to generate enhanced inferences, hybrid subscriptions base upon local and remote subscriptions, and enhance natural language expression evaluation services.

Hybrid client/server architecture for parallel processing

A local feedback mechanism for customizing training models based on user data and directed user feedback is provided in speech recognition applications. The feedback data is filtered at different levels to address privacy concerns for local storage and for submittal to a system developer for enhancement of generic training models.

Local and remote aggregation of feedback data for speech recognition

Architecture for integrating application-dependent information into a constraints component at deployment time or when available. In terms of a general grammar, the constraints component can include or be a general grammar that comprises application-independent information and is structured in such a way that application-dependent information can be integrated into the general grammar without loss of fidelity. The general grammar includes a probability space and reserves a section of the probability space for the integration of application-dependent information. An integration component integrates the application-dependent information into the reserved section of the probability space for recognition processing. The application-dependent information is integrated into the reserved section of the probability space at deployment time or when available. The general grammar is structured to support the integration and improve the overall system.

Application-dependent information for recognition processing

A local text to speech feedback loop is utilized to modify algorithms used in speech synthesis to provide a user with an improved experience. A remote text to speech feedback loop is utilized to aggregate local feedback loop data and incorporate best solutions into new improved text to speech engine for deployment.

Local and remote feedback loop for speech synthesis

Intelligent speech recognition is used to provide users with the ability to utter more user friendly commands. Satisfaction is increased when a user can vocalize a subset of a formal command name and still have the intended command identified and processed. Moreover, greater accuracy in identifying a command application from a user's utterance can be achieved by ignoring command choices associated with unlikely user utterances. An intelligent speech recognition system can identify differing acceptable verbal command phrase forms, e.g., but not limited to, complete commands, command subsequences and command subsets, for different commands supported by the system. Subset blocking words are identified for assistance in reducing the ambiguity in matching user verbal command phrases with valid commands supported by the intelligent speech recognition system.

Intelligent speech recognition of incomplete phrases

A method for compressing multiple dimensional gaussian distributions with diagonal covariance matrixes includes clustering a plurality of gaussian distributions in a multiplicity of clusters for each dimension. Each cluster can be represented by a centroid having a mean and a variance. A total decrease in likelihood of a training dataset is minimized for the representation of the plurality of gaussian distributions.

Method for training of subspace coded gaussian models

A method and apparatus for speaker recognition is provided that matches the noise in training data to noise in testing data using spectral addition. Under spectral addition, the mean and variance for a plurality of frequency components are adjusted in the training data and the test data so that each mean and variance is matched in a resulting matched training signal and matched test signal. The adjustments made to the training data and test data add to the mean and variance of the training data and test data instead of subtracting from the mean and variance.

Method and apparatus using spectral addition for speaker recognition

A method and apparatus for training and using a pattern recognition model are provided. Under the invention, additive noise that matches noise expected in a test signal is included in a training signal. The noisy training signal is passed through one or more noise reduction techniques to produce pseudo-clean training data. The pseudo-clean training data is used to train the pattern recognition model. When the test signal is received, it is passed through the same noise reduction techniques used on the noisy training signal. This produces pseudo-clean test data, which is applied to the pattern recognition model. Under one embodiment, sets of training data are produced with each set containing a different type of noise.

Pattern recognition training method and apparatus using inserted noise followed by noise reduction

Formants, corresponding to input speech units based either on a known text or the results of a speech recognition procedure, are generated from a formant synthesizer. A frequency response is generated based on the synthesized formants. A second frequency response is generated based on a speech signal which is received and which corresponds to utterances of speech units. The synthesized formants are modified based on a comparison of the frequency response corresponding to the synthesized formants and specific proportional characteristics of a frequency response of the input speech signal. In one illustrative embodiment, the comparison is then recalculated and further modifications are made accordingly to improve accuracy. In one illustrative embodiment, time aligning and frequency warping are utilized as modification functions.

System for generating formant tracks by modifying formants synthesized from speech units

The present invention pertains to a concatenative speech synthesis system and method which produces a more natural sounding speech. The system provides for multiple instances of each acoustic unit which can be used to generate a speech waveform representing an linguistic expression. The multiple instances are formed during an analysis or training phase of the synthesis process and are limited to a robust representation of the highest probability instances. The provision of multiple instances enables the synthesizer to select the instance which closely resembles the desired instance thereby eliminating the need to alter the stored instance to match the desired instance. This in essence minimizes the spectral distortion between the boundaries of adjacent instances thereby producing more natural sounding speech.

Growing community of inventors

Seattle, WA, United States of America

Michael D Plumpe