Salesforce Com, Inc.

Stephen Merity

Caiming Xiong

Richard Socher

James Bradbury

Romain Paulus

Ankit Kumar

Ozan Irsoy

Mohit Iyyer

The technology disclosed provides a so-called 'pointer sentinel mixture architecture' for neural network sequence models that has the ability to either reproduce a token from a recent context or produce a token from a predefined vocabulary. In one implementation, a pointer sentinel-LSTM architecture achieves state of the art language modeling performance of 70.9 perplexity on the Penn Treebank dataset, while using far fewer parameters than a standard softmax LSTM.

Pointer sentinel mixture architecture

A novel unified neural network framework, the dynamic memory network, is disclosed. This unified framework reduces every task in natural language processing to a question answering problem over an input sequence. Inputs and questions are used to create and connect deep memory sequences. Answers are then generated based on dynamically retrieved memories.

Dynamic memory network

The technology disclosed provides a quasi-recurrent neural network (QRNN) encoder-decoder model that alternates convolutional layers, which apply in parallel across timesteps, and minimalist recurrent pooling layers that apply in parallel across feature dimensions.

Quasi-recurrent neural network based encoder-decoder model

Growing community of inventors

San Francisco, CA, United States of America

Stephen Merity