Salesforce Com, Inc.

Salesforce, Inc.

Richard Socher

Caiming Xiong

Kazuma Hashimoto

Bryan McCann

James Bradbury

Nitish Shirish Keskar

Victor Zhong

Yingbo Zhou

Jiasen Lu

Stephen Merity

Jia Li

Mingfei Gao

Chien-Sheng Wu

Ehsan Hosseini-Asl

Teresa Anna Marshall

Alexander Rosenberg Johansen

Raffaella Buschiazzo

Bryan Mccann

Chu Hong Hoi

Ankit Chadha

Lichao Sun

Alexander Richard Trott

Kai Sheng Tai

Akari Asai

Stephen Joseph Merity

Lily Hu

Luowei Zhou

Wenhao Liu

Ran Xu

Govardana Sachithanandam Ramachandran

Wojciech Kryscinski

Nazneen Rajani

Shiva Kumar Pentyala

Tong Niu

Romain Paulus

Zeyuan Chen

Xi Victoria Lin

Wenpeng Yin

Ankit Kumar

Wenzhuo Yang

Stephan Zheng

Mridul Gupta

Markus Anderle

Chenxi Li

Harshavardhan Utharavalli

Sewon Min

Jianguo Zhang

Tianmin Shu

Hao Liu

Ozan Irsoy

Indira Iyer

Mohit Iyyer

Latrice Barnett

Simo Arajarvi

Michael Shum

Karim Ahmed

Teresa Marshall

Embodiments are directed to a machine learning recommendation system. The system receives a user query for generating a recommendation for one or more items with an explanation associated with recommending the one or more items. The system obtains first features of at least one user and second features of a set of items. The system provides the first features and the second features to a first machine learning network for determining a predicted score for an item. The system provides a portion of the first features and a portion of the second features to second machine learning networks for determining explainability scores for an item and generating corresponding explanation narratives. The system provides the recommendation for one or more items and corresponding explanation narratives based on ranking predicted scores and explainability scores for the items.

Diversity and explainability parameters for recommendation accuracy in machine learning recommendation systems

The technology disclosed proposes using a combination of computationally cheap, less-accurate bag of words (BoW) model and computationally expensive, more-accurate long short-term memory (LSTM) model to perform natural processing tasks such as sentiment analysis. The use of cheap, less-accurate BoW model is referred to herein as 'skimming'. The use of expensive, more-accurate LSTM model is referred to herein as “reading”. The technology disclosed presents a probability-based guider (PBG). PBG combines the use of BoW model and the LSTM model. PBG uses a probability thresholding strategy to determine, based on the results of the BoW model, whether to invoke the LSTM model for reliably classifying a sentence as positive or negative. The technology disclosed also presents a deep neural network-based decision network (DDN) that is trained to learn the relationship between the BoW model and the LSTM model and to invoke only one of the two models.

Deep neural network-based decision network

A computer-implemented method for dual sequence inference using a neural network model includes generating a codependent representation based on a first input representation of a first sequence and a second input representation of a second sequence using an encoder of the neural network model and generating an inference based on the codependent representation using a decoder of the neural network model. The neural network model includes a plurality of model parameters learned according to a machine learning process. The encoder includes a plurality of coattention layers arranged sequentially, each coattention layer being configured to receive a pair of layer input representations and generate one or more summary representations, and an output layer configured to receive the one or more summary representations from a last layer among the plurality of coattention layers and generate the codependent representation.

Generating dual sequence inferences using a neural network model

The technology disclosed provides a quasi-recurrent neural network (QRNN) encoder-decoder model that alternates convolutional layers, which apply in parallel across timesteps, and minimalist recurrent pooling layers that apply in parallel across feature dimensions.

Quasi-recurrent neural network based encoder-decoder model

Embodiments described herein provide a pipelined natural language question answering system that improves a BERT-based system. Specifically, the natural language question answering system uses a pipeline of neural networks each trained to perform a particular task. The context selection network identifies premium context from context for the question. The question type network identifies the natural language question as a yes, no, or span question and a yes or no answer to the natural language question when the question is a yes or no question. The span extraction model determines an answer span to the natural language question when the question is a span question.

Systems and methods for reading comprehension for a question answering task

A method for using a neural network model for natural language processing (NLP) includes receiving training data associated with a source domain and a target domain; and generating one or more query batches. Each query batch includes one or more source tasks associated with the source domain and one or more target tasks associated with the target domain. For each query batch, class representations are generated for each class in the source domain and the target domain. A query batch loss for the query batch is generated based on the corresponding class representations. An optimization is performed on the neural network model by adjusting its network parameters based on the query batch loss. The optimized neural network model is used to perform one or more new NLP tasks.

System and method for natural language processing using neural network with cross-task training

A system automatically generates recurrent neural network (RNN) architectures for performing specific tasks, for example, machine translation. The system represents RNN architectures using a domain specific language (DSL). The system generates candidate RNN architectures. The system predicts performances of the generated candidate RNN architectures, for example, using a neural network. The system filters the candidate RNN architectures based on their predicted performance. The system generates code for selected a candidate architectures. The generated code represents an RNN that is configured to perform the specific task. The system executes the generated code, for example, to evaluate an RNN or to use the RNN in an application.

Domain specific language for generation of recurrent neural network architectures

A method for sequence-to-sequence prediction using a neural network model includes generating an encoded representation based on an input sequence using an encoder of the neural network model and predicting an output sequence based on the encoded representation using a decoder of the neural network model. The neural network model includes a plurality of model parameters learned according to a machine learning process. At least one of the encoder or the decoder includes a branched attention layer. Each branch of the branched attention layer includes an interdependent scaling node configured to scale an intermediate representation of the branch by a learned scaling parameter. The learned scaling parameter depends on one or more other learned scaling parameters of one or more other interdependent scaling nodes of one or more other branches of the branched attention layer.

Sequence-to-sequence prediction using a neural network model

Embodiments described herein provide safe policy improvement (SPI) in a batch reinforcement learning framework for a task-oriented dialogue. Specifically, a batch reinforcement learning framework for dialogue policy learning is provided, which improves the performance of the dialogue and learns to shape a reward that reasons the invention behind human response rather than just imitating the human demonstration.

Systems and methods for safe policy improvement for task oriented dialogues

Approaches for the translation of structured text include an embedding module for encoding and embedding source text in a first language, an encoder for encoding output of the embedding module, a decoder for iteratively decoding output of the encoder based on generated tokens in translated text from previous iterations, a beam module for constraining output of the decoder with respect to possible embedded tags to include in the translated text for a current iteration using a beam search, and a layer for selecting a token to be included in the translated text for the current iteration. The translated text is in a second language different from the first language. In some embodiments, the approach further includes scoring and pointer modules for selecting the token based on the output of the beam module or copied from the source text or reference text from a training pair best matching the source text.

Systems and methods for structured text translation with tag alignment

The technology disclosed provides a so-called 'joint many-task neural network model' to solve a variety of increasingly complex natural language processing (NLP) tasks using growing depth of layers in a single end-to-end model. The model is successively trained by considering linguistic hierarchies, directly connecting word representations to all model layers, explicitly using predictions in lower tasks, and applying a so-called 'successive regularization' technique to prevent catastrophic forgetting. Three examples of lower level model layers are part-of-speech (POS) tagging layer, chunking layer, and dependency parsing layer. Two examples of higher level model layers are semantic relatedness layer and textual entailment layer. The model achieves the state-of-the-art results on chunking, dependency parsing, semantic relatedness and textual entailment.

Training a joint many-task neural network model using successive regularization

Joint many-task neural network model for multiple natural language processing (NLP) tasks

Embodiments described herein provide methods and systems for training task-oriented dialogue (TOD) language models. In some embodiments, a TOD language model may receive a TOD dataset including a plurality of dialogues and a model input sequence may be generated from the dialogues using a first token prefixed to each user utterance and a second token prefixed to each system response of the dialogues. In some embodiments, the first token or the second token may be randomly replaced with a mask token to generate a masked training sequence and a masked language modeling (MLM) loss may be computed using the masked training sequence. In some embodiments, the TOD language model may be updated based on the MLM loss.

System and methods for training task-oriented dialogue (TOD) language models

Approaches to zero-shot learning include partitioning training data into first and second sets according to classes assigned to the training data, training a prediction module based on the first set to predict a cluster center based on a class label, training a correction module based on the second set and each of the class labels in the first set to generate a correction to a cluster center predicted by the prediction module, presenting a new class label for a new class to the prediction module to predict a new cluster center, presenting the new class label, the predicted new cluster center, and each of the class labels in the first set to the correction module to generate a correction for the predicted new cluster center, augmenting a classifier based on the corrected cluster center for the new class, and classifying input data into the new class using the classifier.

Prediction-correction approach to zero shot learning

Growing community of inventors

Menlo Park, CA, United States of America

Richard Socher