Apple Inc.

Apple Computers Inc.

Yen-Lu Chow

Kai-Fu Lee

Hsiao-Wuen Hon

Kamil A Grajski

Jerome Rene Bellegarda

Erik P Staats

Alejandro Acero

Adam B Fineberg

Peter Vincent deSouza

John W Butzberger

A system and method for processing stroke-based handwriting data for the purposes of automatically scoring and clustering the handwritten data to form letter prototypes. The present invention includes a method for processing digitized stroke-based handwriting data of known character strings, where each of the character strings is represented by a plurality of mathematical feature vectors. In this method, each one of the plurality of feature vectors is labelled as corresponding to a particular character in the character strings. A trajectory is then formed for each one of the plurality of feature vectors labelled as corresponding to a particular character. After the trajectories are formed, a distance value is calculated for each pair of trajectories corresponding to the particular character using dynamic time warping method. The trajectories which are within a sufficiently small distance of each other are grouped to form a plurality of clusters. The clusters are used to define handwriting prototypes which identify subcategories of the character.

Automatic method for scoring and clustering prototypes of handwritten

A method for reducing recognition errors in a speech recognition system that has a user interface, which instructs the user to invoke a new word acquisition module upon a predetermined condition, and that improves the recognition accuracy for poorly recognized words. The user interface of the present invention suggests to a user which unrecognized words may be new words that should be added to the recognition program lexicon. The user interface advises the user to enter words into a new word lexicon that fails to present themselves in an alternative word list for two consecutive tries. A method to improve the recognition accuracy for poorly recognized words via language model adaptation is also provided by the present invention. The present invention increases the unigram probability of an unrecognized word in proportion to the score difference between the unrecognized word and the top one word to guarantee recognition of the same word in a subsequent try. In the event that the score of unrecognized word is unknown (i.e., not in the alternative word list), the present invention increases the unigram probability of the unrecognized word in proportion to the difference between the top one word score and the smallest score in the alternative list.

Method and apparatus for automatically invoking a new word module for

A method of constructing a language model for a phrase-based search in a speech recognition system and an apparatus for constructing and/or searching through the language model. The method includes the step of separating a plurality of phrases into a plurality of words in a prefix word, body word, and suffix word structure. Each of the phrases has a body word and optionally a prefix word and a suffix word. The words are grouped into a plurality of prefix word classes, a plurality of body word classes, and a plurality of suffix word classes in accordance with a set of predetermined linguistic rules. Each of the respective prefix, body, and suffix word classes includes a number of prefix words of same category, a number of body words of same category, and a number of suffix words of same category, respectively. The prefix, body, and suffix word classes are then interconnected together according to the predetermined linguistic rules. A method of organizing a phrase search based on the above-described prefix/body/suffix language model is also described. The words in each of the prefix, body, and suffix classes are organized into a lexical tree structure. A phrase start lexical tree structure is then created for the words of all the prefix classes and the body classes having a word which can start one of the plurality of phrases while still maintaining connections of these prefix and body classes within the language model.

Search engine for phrase recognition based on prefix/body/suffix

A system and method for deriving a large-span semantic language model for a large vocabulary recognition system is disclosed. The method and system maps words from a vocabulary into a vector space, where each word is represented by a vector. After the vectors are mapped to the space, the vectors are clustered into a set of clusters, where each cluster represents a semantic event. After clustering the vectors, a probability that a first word will occur given a history of prior words is computed by (i) calculating a probability that the vector representing the first word belongs to each of the clusters; (ii) calculating a probability of each cluster occurring in a history of prior words; and weighting (i) by (ii) to provide the probability.

Method and system for deriving a large-span semantic language model for

A system for automatic subcharacter unit and lexicon generation for handwriting recognition comprises a processing unit, a handwriting input device, and a memory wherein a segmentation unit, a subcharacter generation unit, a lexicon unit, and a modeling unit reside. The segmentation unit generates feature vectors corresponding to sample characters. The subcharacter generation unit clusters feature vectors and assigns each feature vector associated with a given cluster an identical label. The lexicon unit constructs a lexical graph for each character in a character set. The modeling unit generates a Hidden Markov Model for each set of identically-labeled feature vectors. After a first set of lexical graphs and Hidden Markov Models have been created, the subcharacter generation unit determines for each feature vector which Hidden Markov Model produces a highest likelihood value. The subcharacter generation unit relabels each feature vector according to the highest likelihood value, after which the lexicon unit and the modeling unit generate a new set of lexical graphs and a new set of Hidden Markov models, respectively. The feature vector relabeling, lexicon generation, and Hidden Markov Model generation are performed iteratively until a convergence criterion is met. The final set of Hidden Markov Model model parameters provide a set of subcharacter units for handwriting recognition, where the subcharacter units are derived from information inherent in the sample characters themselves.

System and method for automatic subcharacter unit and lexicon generation

A system and method for performing speaker adaptation in a speech recognition system which includes a set of reference models corresponding to speech data from a plurality of speakers. The speech data is represented by a plurality of acoustic models and corresponding sub-events, and each sub-event includes one or more observations of speech data. A degree of lateral tying is computed between each pair of sub-events, wherein the degree of tying indicates the degree to which a first observation in a first sub-event contributes to the remaining sub-events. When adaptation data from a new speaker becomes available, a new observation from adaptation data is assigned to one of the sub-events. Each of the sub-events is then populated with the observations contained in the assigned sub-event based on the degree of lateral tying that was computed between each pair of sub-events. The reference models corresponding to the populated sub-events are then adapted to account for speech pattern idiosyncrasies of the new speaker, thereby reducing the error rate of the speech recognition system.

Speaker adaptation based on lateral tying for large-vocabulary

The branching decision for each node in a vector quantization (VQ) binary tree is made by a simple comparison of a pre-selected element of the candidate vector with a stored threshold resulting in a binary decision for reaching the next lower level. Each node has a preassigned element and threshold value. Conventional centroid distance training techniques (such as LBG and k-means) are used to establish code-book indices corresponding to a set of VQ centroids. The set of training vectors are used a second time to select a vector element and threshold value at each node that approximately splits the data evenly. After processing the training vectors through the binary tree using threshold decisions, a histogram is generated for each code-book index that represents the number of times a training vector belonging to a given index set appeared at each index. The final quantization is accomplished by processing and then selecting the nearest centroid belonging to that histogram. Accuracy comparable to that achieved by conventional binary tree VQ is realized but with almost a full magnitude increase in processing speed.

Rapid tree-based method for vector quantization

A method of constructing a new active list of phone models from an existing active list of phone models during acoustic matching of a speech recognition system is described. A vector quantized speech vector is compared against each of the phone models in the existing active list to obtain a phone best score for each of the phone models of the existing active list. A best phone best score is determined among all the phone best scores of the phone models to obtain a global best score. A phone model of the phone models from the existing active list is added to the new active list of phone models if the phone best score of that phone model is within a first predetermined value of the global best score. A next phone model of the existing phone of the existing active list is added to the new active list if the phone ending score of that existing phone is within a second predetermined value of a best score of the existing phone model. A next (e.g. first) phone model of a next word of a particular phone model of the existing active list is added to the new active list if the ending score of that particular phone model is within a third predetermined value of the global best score.

Speech recognition system with multi-level pruning for acoustic matching

A method and apparatus for detecting end points of speech activity in an input signal using spectral representation vectors performs beginning point detection using spectral representation vectors for the spectrum of each sample of the input signal and a spectral representation vector for the steady state portion of the input signal. The beginning point of speech is detected when the spectrum diverges from the steady state portion of the input signal. Once the beginning point has been detected, the spectral representation vectors of the input signal are used to determine the ending point of the sound in the signal. The ending point of speech is detected when the spectrum converges towards the steady state portion of the input signal. After both the beginning and ending of the sound are detected, vector quantization distortion can be used to classify the sound as speech or noise.

Method and apparatus for detecting end points of speech activity

A pattern recognition system which continuously adapts reference patterns to more effectively recognize input data from a given source. The input data is converted to a set or series of observed vectors and is compared to a set of Markov Models. The closest matching Model is determined and is recognized as being the input data. Reference vectors which are associated with the selected Model are compared to the observed vectors and updated ('adapted') to better represent or match the observed vectors. This updating method retains the value of these observed vectors in a set of accumulation vectors in order to base future adaptations on a broader data set. When updating, the system also may factor in the values corresponding to neighboring reference vectors that are acoustically similar if the data set from the single reference vector is insufficient for an accurate calculation. Every reference vector is updated after every input; thus reference vectors neighboring an updated reference vector may also be updated. The updated reference vectors are then stored by the computer system for use in recognizing subsequent inputs.

Continuous reference adaptation in a pattern recognition system

A speech recognition system for continuous Mandarin Chinese speech comprises a microphone, an A/D converter, a syllable recognition system, an integrated tone classifier, and a confidence score augmentor. The syllable recognition system generates N-best theories with initial confidence scores. The integrated tone classifier has a pitch estimator to estimate the pitch of the input once and a long-term tone analyzer to segment the estimated pitch according to the syllables of each of the N-best theories. The long-term tone analyzer performs long-term tonal analysis on the segmented, estimated pitch and generates a long-term tonal confidence signal. The confidence score augmentor receives the initial confidence scores and the long-term tonal confidence signals, modifies each initial confidence score according to the corresponding long-term tonal confidence signal, re-ranks the N-best theories according to the augmented confidence scores, and outputs the N-best theories.

Continuous mandarin chinese speech recognition system having an

A method and apparatus for detecting speech activity in an input signal. The present invention includes performing begin point detection using power/zero crossing. Once the begin point has been detected, the present invention uses the cepstrum of the input signal to determine the endpoint of the sound in the signal. After both the beginning and ending of the sound are detected, the present invention uses vector quantization distortion to classify the sound as speech or noise.

Method and apparatus for detecting speech activity using cepstrum vectors

A handwriting signal processing front-end method and apparatus for a handwriting training and recognition system which includes non-uniform segmentation and feature extraction in combination with multiple vector quantization. In a training phase, digitized handwriting samples are partitioned into segments of unequal length. Features are extracted from the segments and are grouped to form feature vectors for each segment. Groups of adjacent from feature vectors are then combined to form input frames. Feature-specific vectors are formed by grouping features of the same type from each of the feature vectors within a frame. Multiple vector quantization is then performed on each feature-specific vector to statistically model the distributions of the vectors for each feature by identifying clusters of the vectors and determining the mean locations of the vectors in the clusters. Each mean location is represented by a codebook symbol and this information is stored in a codebook for each feature. These codebooks are then used to train a recognition system. In the testing phase, where the recognition system is to identify handwriting, digitized test handwriting is first processed as in the training phase to generate feature-specific vectors from input frames. Multiple vector quantization is then performed on each feature-specific vector to represent the feature-specific vector using the codebook symbols that were generated for that feature during training. The resulting series of codebook symbols effects a reduced representation of the sampled handwriting data and is used for subsequent handwriting recognition.

Handwriting signal processing front-end for handwriting recognizers

A speech recognition memory compression method and apparatus subpartitions probability density function (pdf) space along the hidden Markov model (HMM) index into packets of typically 4 to 8 log-pdf values. Vector quantization techniques are applied using a logarithmic distance metric and a probability weighted logarithmic probability space for the splitting of clusters. Experimental results indicate a significant reduction in memory can be obtained with little increase in overall speech recognition error.

Growing community of inventors

Saratoga, CA, United States of America

Yen-Lu Chow