International Business Machines Corporation

Lalit R Bahl

Michael Alan Picheny

Robert L Mercer

Peter V DeSouza

David Nahamoo

Ponani S Gopalakrishnan

Peter V De Souza

Mukund Padmanabhan

Jerome R Bellegarda

Edward Adam Epstein

Peter F Brown

Peter Vincent deSouza

Burn Lewin Lewis

Paul S Cohen

Steven V De Gennaro

Claire Waast-Richard

Jean-Michel Le Roux

Stephane Herman Maes

David M Lubensky

Ramesh Ambat Gopinath

Ellen Marie Eide

John Cocke

Salim Estephan Roukos

Arnold Weinberger

Lazaros Polymenakos

Joachim Hagenauer

John M Lucassen

Frederick Jelinek

Clifton D Cullum, Jr

Steven V DeGennaro

Arthur J Nadas

Ronald Barthold Arps

Peter V Desouza

Mukund Panmanabhan

P S Gopalakrishnan

Peter D De Souza

A method of identifying mismatches between acoustic data and a corresponding transcription, the transcription being expressed in terms of basic units, comprises the steps of: aligning the acoustic data with the corresponding transcription; computing a probability score for each instance of a basic unit in the acoustic data with respect to the transcription; generating a distribution for each basic unit; tagging, as mismatches, instances of a basic unit corresponding to a particular range of scores in the distribution for each basic unit based on a threshold value; and correcting the mismatches.

Identifying mismatches between assumed and actual pronunciations of words

Method for increasing dialect precision and usability in speech recognition and text-to-speech systems

A method for recognizing speech includes the steps of providing a generic model having a baseform representation of a vocabulary of words, identifying a subset of words relating to an application, constructing a task specific model for the subset of words, constructing a composite model by combining the generic and task specific models and modifying the baseform representation of the subset of words such that the subset of words are recognized by the task specific model. A system for recognizing speech includes a composite model having a generic model having a generic baseform representation of a vocabulary of words and a task specific model for recognizing a subset of words relating to an application wherein the subset of words are recognized using a modified baseform representation. A recognizer compares words input thereto with the generic model for words other than the subset of words and with the task specific model for the subset of words.

Specific task composite acoustic models

A technique to improve the recognition accuracy when transcribing speech data that contains data from a wide range of environments. Input data in many situations contains data from a variety of sources in different environments. Such classes include: clean speech, speech corrupted by noise (e.g., music), non-speech (e.g., pure music with no speech), telephone speech, and the identity of a speaker. A technique is described whereby the different classes of data are first automatically identified, and then each class is transcribed by a system that is made specifically for it. The invention also describes a segmentation algorithm that is based on making up an acoustic model that characterizes the data in each class, and then using a dynamic programming algorithm (the viterbi algorithm) to automatically identify segments that belong to each class. The acoustic models are made in a certain feature space, and the invention also describes different feature spaces for use with different classes.

Transcription of speech data with segments from acoustically dissimilar

A system and method are provided which partition the feature space of a classifier by using hyperplanes to construct a binary decision tree or hierarchical data structure for obtaining the class probabilities for a particular feature vector. One objective in the construction of the decision tree is to minimize the average entropy of the empirical class distributions at each successive node or subset, such that the average entropy of the class distributions at the terminal nodes is minimized. First, a linear discriminant vector is computed that maximally separates the classes at any particular node. A threshold is then chosen that can be applied on the value of the projection onto the hyperplane such that all feature vectors that have a projection onto the hyperplane that is less than the threshold are assigned to a child node (say, left child node) and the feature vectors that have a projection greater than or equal to the threshold are assigned to a right child node. The above two steps are then repeated for each child node until the data at a node falls below a predetermined threshold and the node is classified as a terminal node (leaf of the decision tree). After all non-terminal nodes have been processed, the final step is to store a class distribution associated with each terminal node. The class probabilities for a particular feature vector can then be obtained by traversing the decision tree in a top-down fashion until a terminal node is identified which corresponds to the particular feature vector. The information provided by the decision tree is that, in computing the class probabilities for the particular feature vector, only the small number of classes associated with that particular terminal node need be considered. Alternatively, the required class probabilities can be obtained simply by taking the stored distribution of the terminal node associated with the particular feature vector.

System and method for partitioning the feature space of a classifier in

A system and method for recognizing spoken liaisoned words. The method and system identify each word in the vocabulary as a liaison generator and/or liaison receptor. If the word is a liaison receptor, and if the word is preceded by a liaison generator, the most probable recognition result for the word will be the liaison generated by the preceding word plus the word. Liaisons are identified on an immediately preceding word in accordance with rules in a language. A word that ends with an unpronounced consonant phoneme, when followed by a word beginning with a consonant phoneme, and ends with a pronounced phoneme, when followed by a word with a vowel-like phoneme, causes a match list for the current word to be amended with words having liaisons added at their beginnings.

Method for modeling and recognizing speech including word liaisons

Method for performing acoustic model estimation to optimize classification accuracy on speaker derived feature vectors with respect to a plurality of classes corresponding to phones to which a plurality of acoustic models respectively correspond comprises: (a) initializing an acoustic model for each phone; (b) evaluating the merit of the acoustic model initialized for each phone utilizing an objective function having a two component discriminant measure capable of characterizing each phone whereby a first component is defined as a probability that the model for the phone assigns to feature vectors from the phone and a second component is defined as a probability that the model for the phone assigns to feature vectors from other phones; (c) adapting the model for selected phones so as to increase the first component for the phone or decrease the second component for the phone, the adapting step yielding a new model for each selected phone; (d) evaluating the merit of the new models for each phone adapted in step (c) utilizing the two component measure; (e) comparing results of the evaluation of step (b) with results of the evaluation of step (d) for each phone, and if the first component has increased or the second component has decreased, the new model is kept for that phone, else the model originally initialized is kept; (f) estimating parameters associated with each model kept for each phone in order to optimize the function; and (g) evaluating termination criterion to determine if the parameters of the models are optimized.

Apparatus and method for performing model estimation utilizing a

A method and apparatus for using a tree structure to constrain a time-synchronous, fast search for candidate words in an acoustic stream is described. A minimum stay of three frames in each graph node visited is imposed by allowing transitions only every third frame. This constraint enables the simplest possible Markov model for each phoneme while enforcing the desired minimum duration. The fast, time-synchronous search for likely words is done for an entire sentence/utterance. The list of hypotheses beginning at each time frame is stored for providing, on-demand, lists of contender/candidate words to the asynchronous, detailed match phase of decoding.

Method and apparatus for a time-synchronous tree-based search strategy

A method and system of recognizing speech. The method and system perform a fast match on a word in the string of speech to be recognized which generates a fast match list representing words in a system vocabulary that most likely match a current word to be recognized. Next, the method and system perform a detailed match on the words in the fast match list and generate a detailed match list representing words that most likely match the current word to be recognized. Then for each word in the detailed match list that can accept a liaison phoneme from a preceding word, where each word is a liaison receptor, adding to the detailed match list a form of the liaison receptor, where the form represents an addition of a liaison phoneme to the liaison receptor, creating a modified detailed match list which is inclusive of the forms of the liaison receptors added to the detailed match list. Finally the method and system outputs a word in the modified detailed match list that has the highest probability of matching the word to be recognized.

Recognizing speech having word liaisons by adding a phoneme to reference

A system and method for adaptation of a speaker independent speech recognition system for use by a particular user. The system and method gather acoustic characterization data from a test speaker and compare the data with acoustic characterization data generated for a plurality of training speakers. A match score is computed between the test speaker's acoustic characterization for a particular acoustic subspace and each training speaker's acoustic characterization for the same acoustic subspace. The training speakers are ranked for the subspace according to their scores and a new acoustic model is generated for the test speaker based upon the test speaker's acoustic characterization data and the acoustic characterization data of the closest matching training speakers. The process is repeated for each acoustic subspace.

State-dependent speaker clustering for speaker adaptation

A speech coding apparatus in which measured acoustic feature vectors are each represented by the best matched prototype vector. The prototype vectors are generated by storing a model of a training script comprising a series of elementary models. The value of at least one feature of a training utterance of the training script is measured over each of a series of successive time intervals to produce a series of training feature vectors. A first set of training feature vectors corresponding to a first elementary model in the training script is identified. The feature value of each training feature vector signal in the first set is compared to the parameter value of a first reference vector signal to obtain a first closeness score, and is compared to the parameter value of a second reference vector to obtain a second closeness score for each training feature vector. For each training feature vector in the first set, the first closeness score is compared with the second closeness score to obtain a reference match score. A first subset contains those training feature vectors in the first set having reference match scores better than a threshold Q, and a second subset contains those having reference match scores less than the threshold Q. One or more partition values are generated for a first prototype vector frown the first subset of training feature vectors, and one or more additional partition values are generated for the first prototype vector from the second subset of training feature vectors.

Speech coding apparatus having acoustic prototype vectors generated by

The present invention relates to labelling of speech in a context-dependent speech recognition system. When labelling speech using context-dependent prototypes the phone context of a frame of speech needs to be aligned with the appropriate acoustic parameter vector. Since aligning a large amount of data is difficult if based upon arc ranks, the present invention aligns the data using context-independent acoustic prototypes. The phonetic context of each phone of the data is known. Therefore after the alignment step the acoustic parameter vectors are tagged with a corresponding phonetic context. Context-dependent prototype vectors exists for each label. For all labels the context-dependent prototype vectors having the same phonetic context as the tagged acoustic parameter vector are determined. For each label the probability of achieving the tagged acoustic parameter vector is determined given each of the context-dependent label prototype vectors having the same phonetic context as the tagged acoustic parameter vector. The label with the highest probability is associated with the context-dependent acoustic parameter vector.

Labelling speech using context-dependent acoustic prototypes

A speech coding apparatus compares the closeness of the feature value of a feature vector signal of an utterance to the parameter values of prototype vector signals to obtain prototype match scores for the feature vector signal and each prototype vector signal. The speech coding apparatus stores a plurality of speech transition models representing speech transitions. At least one speech transition is represented by a plurality of different models. Each speech transition model has a plurality of model outputs, each comprising a prototype match score for a prototype vector signal. Each model output has an output probability. A model match score for a first feature vector signal and each speech transition model comprises the output probability for at least one prototype match score for the first feature vector signal and a prototype vector signal. A speech transition match score for the first feature vector signal and each speech transition comprises the best model match score for the first feature vector signal and all speech transition models representing the speech transition. The identification value of each speech transition and the speech transition match score for the first feature vector signal and each speech transition are output as a coded utterance representation signal of the first feature vector signal.

Speech recognizer having a speech coder for an acoustic match based on

In speech recognition and speech coding, the values of at least two features of an utterance are measured during a series of time intervals to produce a series of feature vector signals. A plurality of single-dimension prototype vector signals having only one parameter value are stored. At least two single-dimension prototype vector signals having parameter values representing first feature values, and at least two other single-dimension prototype vector signals have parameter values representing second feature values. A plurality of compound-dimension prototype vector signals have unique identification values and comprise one first-dimension and one second-dimension prototype vector signal. At least two compound-dimension prototype vector signals comprise the same first-dimension prototype vector signal. The feature values of each feature vector signal are compared to the parameter values of the compound-dimension prototype vector signals to obtain prototype match scores. The identification values of the compound-dimension prototype vector signals having the best prototype match scores for the feature vectors signals are output as a sequence of coded representations of an utterance to be recognized. A match score, comprising an estimate of the closeness of a match between a speech unit and the sequence of coded representations of the utterance, is generated for each of a plurality of speech units. At least one speech subunit, of one or more best candidate speech units having the best match scores, is displayed.

Speech coding apparatus with single-dimension acoustic prototypes for a

A speech coding apparatus and method for use in a speech recognition apparatus and method. The value of at least one feature of an utterance is measured during each of a series of successive time intervals to produce a series of feature vector signals representing the feature values. A plurality of prototype vector signals, each having at least one parameter value and a unique identification value are stored. The closeness of the feature vector signal is compared to the parameter values of the prototype vector signals to obtain prototype match scores for the feature value signal and each prototype vector signal. The identification value of the prototype vector signal having the best prototype match score is output as a coded representation signal of the feature vector signal. Speaker-dependent prototype vector signals are generated from both synthesized training vector signals and measured training vector signals. The synthesized training vector signals are transformed reference feature vector signals representing the values of features of one or more utterances of one or more speakers in a reference set of speakers. The measured training feature vector signals represent the values of features of one or more utterances of a new speaker/user not in the reference set.

Growing community of inventors

Amawalk, NY, United States of America

Lalit R Bahl