International Business Machines Corporation

Mukund Padmanabhan

David Nahamoo

Lalit R Bahl

George Andrei Saon

Michael Alan Picheny

Stephane Herman Maes

Ponani S Gopalakrishnan

Salim Estephan Roukos

Geoffrey G Zweig

Dimitri Kanevsky

Jing Chang Huang

Arthur R Zingher

David M Lubensky

Yuqing Gao

Ramesh Ambat Gopinath

Jeffrey Scott Sorensen

Lidia Mangu

Satyanarayana Dharanipragada

Peter Vincent deSouza

In connection with speech recognition, the design of a linear transformation &#x3b8;&#x3b5;, of rank p&#xd7;n, which projects the features of a classifier x&#x3b5;onto y=&#x3b8;x&#x3b5;such as to achieve minimum Bayes error (or probability of misclassification). Two avenues are explored: the first is to maximize the &#x3b8;-average divergence between the class densities and the second is to minimize the union Bhattacharyya bound in the range of &#x3b8;. While both approaches yield similar performance in practice, they outperform standard linear discriminant analysis features and show a 10% relative improvement in the word error rate over known cepstral features on a large vocabulary telephony speech recognition task.

Minimum bayes error feature selection in speech recognition

Methods and arrangements using lattice-based information for unsupervised speaker adaptation. By performing adaptation against a word lattice, correct models are more likely to be used in estimating a transform. Further, a particular type of lattice proposed herein enables the use of a natural confidence measure given by the posterior occupancy probability of a state, that is, the statistics of a particular state will be updated with the current frame only if the a posteriori probability of the state at that particular time is greater than a predetermined threshold.

Lattice-based unsupervised maximum likelihood linear regression for speaker adaptation

Methods and apparatus are provided for processing an information signal containing content presented in accordance with at least one modality. In one aspect of the present invention, a method of processing an information signal containing content presented in accordance with at least one modality, comprises the steps of: (i) obtaining the information signal; (ii) performing content detection on the information signal to detect whether the information signal includes particular content presented in accordance with the at least one modality; and (iii) generating a control signal, when the particular content is detected, for use in controlling a rendering property of the particular content and/or implementation of a specific action relating to the particular content. Various illustrative embodiments in the context of speech signal processing for use in voicemail and/or cellular phone applications are provided, as well as illustrative embodiments associated with the processing of multi-modal or multimedia information signals. Also, the present invention provides for storing selectively marked information, even in the absence of content detection, such that the information may be rendered and/or used at a later time. The invention also extends to processing of text-based and markup language-based signals, e.g., XML documents.

Method and apparatus for processing information signals based on content

Generally, the present invention determines and uses spectral peak information, which preferably augments feature vectors and creates augmented feature vectors. The augmented feature vectors decrease errors in pattern recognition, increase noise immunity for wide-band noise, and reduce reliance on noisy formant features. Illustratively, one way of determining spectral peak information is to split pattern data into a number of frequency ranges and determine spectral peak information for each of the frequency ranges. This allows single peak selection. All of the spectral peak information is then used to augment a feature vector. Another way of determining spectral peak information is to use an adaptive Infinite Impulse Response filter to provide this information. Additionally, the present invention can determine and use incremental information. The incremental information is relatively easy to calculate and helps to determine if additional or changed features are worthwhile. The incremental information is preferably determined by determining a difference between mutual information (between the feature vector and the classes to be disambiguated) for new or changed feature vectors and mutual information for old feature vectors.

Determination and use of spectral peak information and incremental information in pattern recognition

Techniques are described for decreasing the number of errors when consensus decoding is used during speech recognition. A number of corrective rules are applied to confusion sets that are extracted during real-time speech recognition. The corrective rules are determined during training of the speech recognition system, which entails using many training confusion sets. A learning process is used that generates a number of possible rules, called template rules, that can be applied to the training confusion sets. The learning process also determines the corrective rules from the template rules. The corrective rules operate on the real-time confusion sets to select hypothesis words from the confusion sets, where the hypothesis words are not necessarily the words having the highest score.

Error corrective mechanisms for consensus decoding of speech

Techniques are provided for enumerating regularly identifiable or stereotypical phrases that people commonly use to convey particular information, and where exactly in these phrases the particular information is to be found. In one embodiment, such phrases are referred to as 'regular expressions.' Using such enumerated phrases, the invention is able to automatically identify them in an input data stream and then identify and extract the particular information associated with the phrase that is being sought, e.g., important or relevant information.

Information extraction from documents with regular expression matching

The present invention provides a new approach to heteroscedastic linear discriminant analysis (HDA) by defining an objective function which maximizes the class discrimination in the projected subspace while ignoring the rejected dimensions. Moreover, we present a link between discrimination and the likelihood of the projected samples and show that HDA can be viewed as a constrained maximum likelihood (ML) projection for a full covariance gaussian model, the constraint being given by the maximization of the projected between-class scatter volume. The present invention also provides that, under diagonal covariance gaussian modeling constraints, applying a diagonalizing linear transformation (e.g., MLLT&mdash;maximum likelihood linear transformation) to the HDA space results in an increased classification accuracy. In another embodiment, the heteroscedastic discriminant objective function assumes that models associated with the function have diagonal covariances thereby resulting in a diagonal heteroscedastic discriminant objective function.

Methods and apparatus for performing heteroscedastic discriminant analysis in pattern recognition systems

An archive system for records with an audio component, which uses automated speech recognition to create a multi-layered archive pyramid. The archive pyramid includes successive layers of data stored at varying data rates such as original video data, compressed video data, original audio, compressed audio data, recognized word-lattices, recognized word-bags and a global word index. The disclosed system uses automatic speech recognition to transcribe from audio to searchable index layers. During a search operation, automatic and semi-automatic techniques are used to search the archive pyramid from the smallest narrowest layers to the largest widest layers, to identify a moderate subset of records. This subset is further refined by a manual survey of regenerated compressed audio. Finally, the selected records are retrieved from the original audio archive layer.

Audio/video archive system and method for automatic indexing and searching

A method of adapting a speech recognition system to one or more acoustic conditions comprises the steps of: (i) computing cumulative distribution functions based on dimensions of speech vectors associated with training speech data provided to the speech recognition system; (ii) computing cumulative distribution functions based on dimensions of speech vectors associated with test speech data provided to the speech recognition system; (iii) computing a nonlinear transformation mapping based on the cumulative distribution functions associated with the training speech data and the cumulative distribution functions associated with the test speech data; and (iv) applying the nonlinear transformation mapping to speech vectors associated with the test speech data prior to recognition, wherein the speech vectors transformed in accordance with the nonlinear transformation mapping are substantially similar to speech vectors associated with the training speech data.

Method and apparatus for rapid adapt via cumulative distribution function matching for continuous speech

A method of performing speaker adaptation of acoustic models in a band-quantized speech recognition system, wherein the system including one or more acoustic models represented by a feature space of multi-dimensional gaussians, whose dimensions are partitioned into bands, and the gaussian means and covariances within each band are quantized into atoms, comprises the following steps. A decoded segment of a speech signal associated with a particular speaker is obtained. Then, at least one adaptation mapping based on the decoded segment is computed. Lastly, the at least one adaptation mapping is applied to the atoms of the acoustic models to generate one or more acoustic models adapted to the particular speaker. Accordingly, a fast speaker adaptation methodology is provided for use in real-time applications.

Methods and apparatus for fast adaptation of a band-quantized speech decoding system

A method of forming an augmented textual training corpus with compound words for use with an associated with a speech recognition system includes computing a measure for a consecutive word pair in the training corpus. The measure is then compared to a threshold value. The consecutive word pair is replaced in the training corpus with a corresponding compound word depending on the result of the comparison between the measure and the threshold value. One or more measures may be employed. A first measure is an average of a direct bigram probability value and a reverse bigram probability value. A second measure is based on mutual information between the words in the pair. A third measure is based on a comparison of the number of times a co-articulated baseform for the pair is preferred over a concatenation of non-co-articulated individual baseforms of the words forming the pair. A fourth measure is based on a difference between an average phone recognition score for a particular compound word and a sum of respective average phone recognition scores of the words of the pair.

Methods and apparatus for forming compound words for use in a continuous speech recognition system

A method of identifying mismatches between acoustic data and a corresponding transcription, the transcription being expressed in terms of basic units, comprises the steps of: aligning the acoustic data with the corresponding transcription; computing a probability score for each instance of a basic unit in the acoustic data with respect to the transcription; generating a distribution for each basic unit; tagging, as mismatches, instances of a basic unit corresponding to a particular range of scores in the distribution for each basic unit based on a threshold value; and correcting the mismatches.

Identifying mismatches between assumed and actual pronunciations of words

A method for recognizing speech includes the steps of providing a generic model having a baseform representation of a vocabulary of words, identifying a subset of words relating to an application, constructing a task specific model for the subset of words, constructing a composite model by combining the generic and task specific models and modifying the baseform representation of the subset of words such that the subset of words are recognized by the task specific model. A system for recognizing speech includes a composite model having a generic model having a generic baseform representation of a vocabulary of words and a task specific model for recognizing a subset of words relating to an application wherein the subset of words are recognized using a modified baseform representation. A recognizer compares words input thereto with the generic model for words other than the subset of words and with the task specific model for the subset of words.

Specific task composite acoustic models

A messaging system for receiving speech over a telephone and converting the speech to text includes a first server for receiving speech input by a user, a speech recognition system for converting the speech to text, a speech synthesizer for converting the text to speech for playing back the synthesized speech for correction by the user and a correction mechanism for enabling the user to correct the speech such that the corrected speech is provided as text for transmittal over a communication system.

Telephone messaging and editing system

A method of speech recognition, in accordance with the present invention includes the steps of grouping acoustics to form classes based on acoustic features, clustering training speakers by the classes to provide class-specific cluster systems, selecting from the cluster systems, a subset of cluster systems closest to adaptation data from a test speaker, transforming the subset of cluster systems to bring the subset of cluster systems closer to the test speaker based on the adaptation data to form adapted cluster systems and combining the adapted cluster systems to create a speaker adapted system for decoding speech from the test speaker. System and methods for building speech recognition systems as well as adapting speaker systems for class-specific speaker clusters are included.

Growing community of inventors

White Plains, NY, United States of America

Mukund Padmanabhan