Salesforce Com, Inc.

Salesforce, Inc.

Semih Yavuz

Yingbo Zhou

Caiming Xiong

Rui Meng

Kazuma Hashimoto

Nitish Shirish Keskar

Wojciech Kryscinski

Nazneen Rajani

Huan Wang

Tong Niu

Xi Victoria Lin

Bo Pang

Qingyun Wang

Haopeng Zheng

Ye Liu

Meghana Bhat

Embodiments described herein provide a system for selecting a neural network based natural language processing (NLP) model for building a custom artificial intelligence (AI) stack for a user. The system includes a communication interface that established connections to one or more external servers hosting one or more neural network based NLP models, a memory; and a processor executing operations including: selecting a source document based on a custom NLP application; generating, by a first language model, a summary of the source document; generating, by a second language model, one or more questions based on at least one of the summary or the source document; transmitting, via the communication interface, the one or more questions to the one or more neural network based NLP models; receiving, via the communication interface, one or more answers generated by the one or more neural network based NLP models.

Systems and methods for selecting neural network models for building a custom artificial intelligence stack

Embodiments described herein provide a prompt-based transfer learning method that employs shared latent space prompt tuning). Specifically, a shared latent space is assumed, among all source and target tasks, where each vector in the space captures a basis skill to do a particular task. Given an instance (from either a source task or a target task), it is first encoded into an instance representation vector and then queries the latent space, which yields a skill vector for this instance. This vector modulates a frozen model, via soft prompts which are a simple prompt transformation (the prompt generator in FIG.) of the basis skill vector, to generate an answer for the instance. The latent space and prompt transformation are learned end-to-end in upstream pre-training on source tasks.

Systems and methods for shared latent space prompt tuning

Embodiments described herein provide a semantic parsing framework which may be referred to as Uni-Parser. The Uni-Parser framework may be applied to question answering on both knowledge bases and databases. The three main stages of the Uni-Parser framework are enumeration, ranking, and generation. At the enumeration stage, primitives are enumerated based on matching the question to the data structure. After enumerating primitives, the Uni-Parser framework may rank the primitives used a trained ranker model. The top ranked primitives may then be used as inputs to a generator which is a learned sequence to sequence model which produces a logical form.

Systems and methods for semantic parsing with primitive level enumeration

Embodiments described herein provide dynamic blocking, a decoding algorithm which enables large-scale pretrained language models to generate high-quality paraphrases in an un-supervised setting. Specifically, in order to obtain an alternative surface form, when the language model emits a token that is present in the source sequence, the language model is prevented from generating the next token that is the same as the subsequent source token in the source sequence at the next time step. In this way, the language model is forced to generate a paraphrased sequence of the input source sequence, but with mostly different wording.

Systems and methods for unsupervised paraphrase generation

Embodiments described herein provide document summarization systems and methods that utilize fine-tuning of pre-trained abstractive summarization models to produce summaries that more faithfully track the content of the documents. Such abstractive summarization models may be pre-trained using a corpus consisting of pairs of articles and associated summaries. For each article-summary pair, a pseudo label or control code is generated and represents a faithfulness of the summary with respect to the article. The pre-trained model is then fine-tuned based on the article-summary pairs and the corresponding control codes. The resulting fine-tuned models then provide improved faithfulness in document summarization tasks.

Systems and methods for abstractive document summarization with entity coverage control

Embodiments described herein provide systems and methods for data-to-text generation. The embodiments receive input data that includes a resource description framework (RDF) triples in an RDF graph. A data-to-text generation system generates position aware embeddings, including position embeddings, triple role embeddings, and tree-level embeddings. Using the position aware embeddings and the RDF graph, the data-to-text generation system generates a textual description for the RDF graph.

Structured graph-to-text generation with two step fine-tuning

Growing community of inventors

Redwood City, CA, United States of America

Semih Yavuz