Microsoft Technology Licensing, LLC

Google Inc.

Other

Microsoft Corporation

Robert C Moore

Christopher Brian Quirk

Arul A Menezes

Eric David Brill

Eric K Ringger

Richard F Rashid

Michael Gamon

Simon H Corston-Oliver

Joshua T Goodman

Stephen D Richardson

Anthony Aue

Mark Edward Johnson

Zhu Zhang

Martine Smets

Steven D Richardson

Simon Corston-Oliver

Techniques of creating a classifier model for a multi-class linear classifier are disclosed. The classifier model includes feature weights for each of a plurality of feature and label combinations. One technique includes selecting an update set of feature weights to be updated from the classifier model, determining an update for each of the feature weights in the selected update set using a processor, a plurality of the updated determined independently of all other updates and determined based on a largest reduction in an output of a loss function, modifying each of the updates using a step size, and updating the classifier model feature weights using the modified, determined updates.

Feature weight training techniques

The claimed subject matter provides systems and/or methods for training feature weights in a statistical machine translation model. The system can include components that obtain lists of translation hypotheses and associated feature values, set a current point in the multidimensional feature weight space to an initial value, chooses a line in the feature weight space that passes through the current point, and resets the current point to optimize the feature weights with respect to the line. The system can further include components that set the current point to be a best point attained, reduce the list of translation hypotheses based on a determination that a particular hypothesis has never been touched in optimizing the feature weights from at least one of an initial staring point or a randomly selected restarting point, and output the point ascertained to be the best point in the feature weight space.

Faster minimum error rate training for weighted linear models

Described is a technology by which a probability is estimated for a token in a sequence of tokens based upon a number of zero or more times (actual counts) that the sequence was observed in training data. The token may be a word in a word sequence, and the estimated probability may be used in a statistical language model. A discount parameter is set independently of interpolation parameters. If the sequence was observed at least once in the training data, a discount probability and an interpolation probability are computed and summed to provide the estimated probability. If the sequence was not observed, the probability is estimated by computing a backoff probability. Also described are various ways to obtain the discount parameter and interpolation parameters.

N-gram model smoothing with independently controllable parameters

A method of translation includes uploading a source text portion to a back end processor. The back end processor identifies a subset of translation knowledge associated with the source text portion. The back end processor downloads the subset to a front end processor. A translation engine runs on the front end processor. The translation engine generates a translation of the source text portion as a function of the subset.

Machine translation split between front end and back end processors

Described is a technology by which a statistical N-gram (e.g., language) model is trained using an N-gram selection technique that helps reduce the size of the final N-gram model. During training, a higher-order probability estimate for an N-gram is only added to the model when the training data justifies adding the estimate. To this end, if a backoff probability estimate is within a maximum likelihood set determined by that N-gram and the N-gram's associated context, or is between the higher-order estimate and the maximum likelihood set, then the higher-order estimate is not included in the model. The backoff probability estimate may be determined via an iterative process such that the backoff probability estimate is based on the final model rather than any lower-order model. Also described is additional pruning referred to as modified weighted difference pruning.

N-gram selection for practical-sized language models

Techniques of line-searching to find a minimum value of a loss function l projected onto a line are disclosed, wherein the loss function l is continuous and piece-wise linear. One technique includes identifying points for at least one output derived from the loss function l at which the slope of the output changes, determining at least one derivative value for an initial point and at least one change in derivative value for at least some of the points using a processor, determining a cumulative reduction in loss for at least some of the points using at least one of the determined derivative values or at least one of the change in derivative values, and selecting a point corresponding to the minimum value of the loss function l based on the cumulative reduction in loss.

Line searching techniques

Improved systems, methods and apparatuses are provided for fast beam-search decoding for phrasal statistical machine translation. The provided techniques incorporate a front-loaded distortion penalty estimate for future estimated distortion penalty and/or early pruning to reduce the search space. The improvements result in up to an order of magnitude increase in translation speed for statistical machine translation systems. The disclosed details enable various refinements and modifications according to decoder and system design considerations.

Fast beam-search decoding for phrasal statistical machine translation

A phrase translation model is trained without assuming a segmentation of training data into non-overlapping phrase pairs. Instead, the training algorithm assumes that any particular phrase instance has only a single phrase instance in another language as its translation in that instance, but that phrases can overlap. The model is trained by computing expected phrase alignment counts, deriving selection probabilities from current estimates of translation probabilities and then re-estimating phrase translation probabilities according to the expected phrase alignment counts computed. The model is trained by iterating over these steps until one or more desired stopping criteria are reached. The trained model can be deployed in a machine translation system.

Generating a phrase translation model by iteratively estimating phrase translation probabilities

Dependency grammars are transformed to context-free grammars. The context-free grammars can be used in a parser to parse input sentences and identify relationships among words in the sentence.

Converting dependency grammars to efficiently parsable context-free grammars

A weighted linear word alignment model linearly combines weighted features to score a word alignment for a bilingual, aligned pair of text fragments. The features are each weighted by a feature weight. One of the features is a word association metric, which may be generated from surface statistics.

Weighted linear bilingual word alignment model

A method is provided for identifying phrase alignment pairs between a source sentence and a target sentence. Boundaries for a phrase in the source sentence are identified by requiring that a source word be aligned with at least one target word in a target sentence in order to form a boundary for the source phrase. Boundaries for a phrase in the target sentence are identified based on alignments between words in the source phrase and words in the target sentence. The words in the target phrase are examined to determine if any of the words are aligned with source words outside of the source phrase. If they are not aligned with source words outside of the source phrase, the source phrase and target phrase are determined to form an alignment pair and are stored as a phrase alignment pair.

Efficient phrase pair extraction from bilingual word alignments

A pair of bilingual, aligned text fragments are received and word type alignment is performed on the text fragments. Word token alignment is then performed.

Association-based bilingual word alignment

A method of decoding an input semantic structure to generate an output semantic structure. A set of transfer mappings are provided. A score is calculated for at least one transfer mapping in the set of transfer mappings using a statistical model. At least one transfer mapping is selected based on the score and used to construct the output semantic structure.

Statistical language model for logical form using transfer mappings

The present invention is a sentence realization system that processes an abstract linguistic representation (ALR) of a sentence into a structure that can be fully realizable. The system includes a tree conversion component that receives the ALR and generates a basic syntax tree from the ALR. A global movement component then receives the basic syntax tree and hierarchically orders child nodes in that syntax tree relative to ancestor nodes. An intra-constituent ordering component then establishes a linear order among the nodes such that the syntax tree is fully ordered. A surface cleanup component receives the fully ordered tree and performs a number of realization operations to generate surface realizations for constituents that are still represented in an abstract way in the fully ordered syntax tree.

Growing community of inventors

Mercer Island, WA, United States of America

Robert C Moore