Google Inc.

Slav Petrov

Ryan T McDonald

Dipanjan Das

Alexander Matthew Rush

Keith Hall

Yonghui Wu

Amarnag Subramanya

Andrew M Dai

Fernando Pereira

Melvin Jose Johnson Premkumar

Orhan Firat

Yanping Huang

Yuanzhong Xu

David Richard So

Maxim Krikun

Aurko Roy

Nan Du

Rohan Anil

Gaurav Mishra

Xavier Eduardo Garcia

Oscar Carl Tackstrom

Dmitry Lepikhin

Siamak Shakeri

Yong Cheng

Jonathan Hudson Clark

Erica Ann Moreira

Yujing Zhang

Joakim Nivre

Zachary Alexander Nado

Kefan Xiao

Jin Young Sohn

Eric Jun Jie Ni

Vladimir Feinberg

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network to perform any one or more of a variety of machine learning tasks. For example, the neural network can be configured as a generative neural network, e.g., an autoregressive generative neural network.

Training of large neural networks

A computer-implemented method can include receiving a speech input representing a question, converting the speech input to a string of characters, and obtaining tokens each representing a potential word. The method can include determining one or more part-of-speech (POS) tags for each token and determining sequences of the POS tags for the tokens, each sequence of the POS tags including one POS tag per token. The method can include determining one or more parses for each sequence of the POS tags for the tokens and determining a most-likely parse and its corresponding sequence of the POS tags for the tokens to obtain a selected parse and a selected sequence of the POS tags for the tokens. The method can also include determining a most-likely answer to the question using the selected parse and the selected sequence of the POS tags for the tokens and outputting the most-likely answer.

Techniques for discriminative dependency parsing

A method and system are provided for a part-of-speech tagger that may be particularly useful for resource-poor languages. Use of manually constructed tag dictionaries from dictionaries via bitext can be used as type constraints to overcome the scarcity of annotated data in some instances. Additional token constraints can be projected from a resource-rich source language via word-aligned bitext. Several example models are provided to demonstrate this such as a partially observed conditional random field model, where coupled token and type constraints may provide a partial signal for training. The disclosed method achieves a significant relative error reduction over the prior state of the art.

Weakly supervised part-of-speech tagging with coupled token and type constraints

A source language sentence is tagged with non-lexical tags, such as part-of-speech tags and is parsed using a lexicalized parser trained in the source language. A target language sentence that is a translation of the source language sentence is tagged with non-lexical labels (e.g., part-of speech tags) and is parsed using a delexicalized parser that has been trained in the source language to produce k-best parses. The best parse is selected based on the parse's alignment with lexicalized parse of the source language sentence. The selected best parse can be used to update the parameter vector of a lexicalized parser for the target language.

Multi-source transfer of delexicalized dependency parsers

A dependency parsing method can include determining an index set of possible head-modifier dependencies for a sentence. The index set can include inner arcs and outer arcs, inners arcs representing possible dependency between words in the sentence separated by a distance less than or equal to a threshold and outer arcs representing possible dependency between words in the sentence separated by a distance greater than the threshold. The index set can be pruned to include: (i) each specific inner arc when a likelihood that the specific inner arc is appropriate is greater than a first threshold, and (ii) the outer arcs when a likelihood that there exists any possible outer arc that is appropriate is greater than the first threshold. The method can include further pruning the pruned index set based on a second parsing algorithm, and determining a most-likely parse for the sentence from the pruned index set.

Efficient parsing with structured prediction cascades

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for graph-based semi-supervised learning of structured tagging models. In one aspect, a method includes creating a graph having a plurality of unique vertices in which vertices in a first set of vertices represent n-grams that are each associated with a respective part-of-speech and that were derived from labeled source domain text, and in which vertices in a different second set of vertices represent n-grams that are not associated with a part-of-speech and that were derived from unlabeled target domain text. A respective measure of similarity is calculated between the vertices in each of the pairs based at least partially on a distance between the respective features of the pair used to weight a graph edge between the pair.

Graph-based semi-supervised learning of structured tagging models

Growing community of inventors

New York, NY, United States of America

Slav Petrov