Sri International

Google Inc.

Shalini Ghosh

Patrick Denis Lincoln

Oriol Vinyals

Larry Paul Heck

Bhaskar S Ramamurthy

Brian Strope

Thomas L Dean

Howard Scott Roy

Susmit Jha

Ashish Tiwari

In general, the disclosure describes techniques for facilitating trust in neural networks using a trusted neural network system. For example, described herein are multi-headed, trusted neural network systems that can be trained to satisfy one or more constraints as part of the training process, where such constraints may take the form of one or more logical rules and cause the objective function of at least one the heads of the trusted neural network system to steer, during machine learning model training, the overall objective function for the system toward an optimal solution that satisfies the constraints. The constraints may be non-temporal, temporal, or a combination of non-temporal and temporal. The constraints may be directly compiled to a neural network or otherwise used to train the machine learning model.

Trusted neural network system

Methods and systems of using machine learning to create a trusted model that improves the operation of a computer system controller are provided herein. In some embodiments, a machine learning method includes training a model using input data, extracting the model, and determining whether the model satisfies the trust-related constraints. If the model does not satisfy the trust-related constraint, modifying at least one of: the model using one or more model repair algorithms, the input data using one or more data repair algorithms, or a reward function of the model using one or more reward repair algorithms, and re-training the model using at least one of the modified model, the modified input data, or the modified reward function. If the model satisfies the trust-related constraints, providing the model as a trusted model that enables a computer system controller to perform system actions within predetermined guarantees.

Systems and methods for machine learning using a trusted model

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing word sequences using neural networks. One of the methods includes receiving a first sequence of words arranged according to a first order; and for each word in the first sequence, beginning with a first word in the first order: determining a topic vector that is associated with the word; generating a combined input from the word and the topic vector, and processing the combined input through one or more sequence modeling layers to generate a sequence modeling output for the word; and processing one or more of the sequence modeling outputs through an output layer to generate a neural network output for the first sequence of words.

Topic-based sequence modeling neural networks

Growing community of inventors

Redwood City, CA, United States of America

Shalini Ghosh