Salesforce, Inc.

Salesforce Com, Inc.

Yingbo Zhou

Caiming Xiong

Semih Yavuz

Richard Socher

Kazuma Hashimoto

Huan Wang

Wenhao Liu

Tian Xie

Xinyi Yang

Jin Qu

Ehsan Hosseini-Asl

Feihong Wu

Rui Meng

Bo Pang

Govardana Sachithanandam Ramachandran

Tong Niu

Weiran Wang

Young Mo Kang

Ye Liu

Xugang Ye

Erik Lennart Nijkamp

Luowei Zhou

Ran Xu

Nitish Shirish Keskar

Jia Li

Wojciech Kryscinski

Mingfei Gao

Ning Yu

Lichao Sun

Ganeswara Rao Dulam

Lav Raj Varshney

Shafiq Rayhan Joty

Meghana Bhat

Jiacheng Xu

Susan Putvin

Hari Krishna Pottabathula

Madhuri Gore

Junwen Bai

Hiroaki Hayashi

Devansh Arpit

Haopeng Zheng

Xilai Li

Embodiments described herein provide a method for building an artificial intelligence (AI) agent to respond to a user query. The method includes: receiving a user query; retrieving a set of documents that are ranked based on respective relevance scores of a first type to the user query; generating a core question that filters out irrelevant texts from the user query; generating a first summary of a first document from the set of documents and a first reasoning output explaining how the first summary addresses the core question; generating a relevance score of a second type and a corresponding reranking for the first document based at least in part on a combination of the core question and the first reasoning output; generating a response to the user query using one or more top-ranked documents according to generated rerankings of the set of documents.

Systems and methods for a reasoning-intensive reranking based artificial intelligence conversation agent

Embodiments described herein provide systems and methods for training a text retrieval model. A system may generate queries associated with provided documents. The queries may be generated in one or more different manners. Examples of query generation may include extracting relevant spans of text from the documents, prompting a language model for a topic, title, abstractive summary, and/or extractive summary based on the documents. Metadata such as title or other HTML tags may be used as queries. Using the one or more queries, the text retrieval model may be trained using contrastive learning, using the generated query, and positive and negative sample documents. A fine-tuning training phase may be performed using domain-specific data which may also be done with generated query pairs, or may be done in a supervised fashion with provided queries. The text retrieval model may be used to locate documents given an input query.

Systems and methods for unsupervised training in text retrieval tasks

Embodiments described herein provide a system for selecting a neural network based natural language processing (NLP) model for building a custom artificial intelligence (AI) stack for a user. The system includes a communication interface that established connections to one or more external servers hosting one or more neural network based NLP models, a memory; and a processor executing operations including: selecting a source document based on a custom NLP application; generating, by a first language model, a summary of the source document; generating, by a second language model, one or more questions based on at least one of the summary or the source document; transmitting, via the communication interface, the one or more questions to the one or more neural network based NLP models; receiving, via the communication interface, one or more answers generated by the one or more neural network based NLP models.

Systems and methods for selecting neural network models for building a custom artificial intelligence stack

Embodiments described herein provide regression testing using artificial intelligence. A regression testing network model for a first plurality of organizations using a common codebase is provided. The regression testing network model provides an organization finite state machine (FSM) model for each organization. A first dataset including samples of the organization FSM models based on regression testing for one or more previous releases of the common codebase prior to a first release of the common codebase is received. A training dataset is generated based on the first dataset. The regression testing network model using the training dataset. A second plurality of organizations for regression testing for the first release is determined, from the first plurality of organizations, using the trained regression testing network model.

Systems and methods for regression testing using artificial intelligence

Embodiments described herein provide systems and methods for text generation using a natural language processing model. Natural language input is received and encoded by an encoder. A decoder generates a plurality of next-node probabilities for a set of candidate nodes that are next to at least one of K previously decoded paths of nodes on a directed search graph. Scores are computed for each of the candidate nodes and done in parallel among the K previously decoded paths. The highest scoring K nodes are selected for addition to the directed search graph. The search continues until a search budget is exhausted. Generated sequences are output. These sequences may represent a plurality of sentences or questions, depending on the task.

Systems and methods for search based neural text generation models

A computing device may perform training of a set of machine learning models on a first data set associated with a first domain. In some examples, the training may include, for each machine learning model of the set of machine learning models, inputting, as values for a set of parameters of the respective sets of parameters and for an iteration of a set of iterations, a moving average of the set of parameters calculated over a threshold number of previous iterations. The computing device may select a set of model states that are generated during the training of the plurality of machine learning models based on a validation performance of the set of model states performed during the training. The computing device may then generate an ensembled machine learning model by aggregating the set of machine learning models corresponding to the set of selected model states.

Techniques for machine learning model selection for domain generalization

Embodiments described herein provide a prompt-based transfer learning method that employs shared latent space prompt tuning). Specifically, a shared latent space is assumed, among all source and target tasks, where each vector in the space captures a basis skill to do a particular task. Given an instance (from either a source task or a target task), it is first encoded into an instance representation vector and then queries the latent space, which yields a skill vector for this instance. This vector modulates a frozen model, via soft prompts which are a simple prompt transformation (the prompt generator in FIG.) of the basis skill vector, to generate an answer for the instance. The latent space and prompt transformation are learned end-to-end in upstream pre-training on source tasks.

Systems and methods for shared latent space prompt tuning

Embodiments described herein provide systems and methods for a partially supervised training model for online action detection. Specifically, the online action detection framework may include two modules that are trained jointly—a Temporal Proposal Generator (TPG) and an Online Action Recognizer (OAR). In the training phase, OAR performs both online per-frame action recognition and start point detection. At the same time, TPG generates class-wise temporal action proposals serving as noisy supervisions for OAR. TPG is then optimized with the video-level annotations. In this way, the online action detection framework can be trained with video-category labels only without pre-annotated segment-level boundary labels.

Systems and methods for partially supervised online action detection in untrimmed videos

Embodiments described herein combine both masked reconstruction and predictive coding. Specifically, unlike contrastive learning, the mutual information between past states and future states are directly estimated. The context information can also be directly captured via shifted masked reconstruction—unlike standard masked reconstruction, the target reconstructed observations are shifted slightly towards the future to incorporate more predictability. The estimated mutual information and shifted masked reconstruction loss can then be combined as the loss function to update the neural model.

Systems and methods for mutual information based self-supervised learning

Embodiments described herein provide a cross-lingual sentence alignment framework that is trained only on rich-resource language pairs. To obtain an accurate aligner, a pretrained multi-lingual language model is used, and a classifier is trained on parallel data from rich-resource language pairs. This trained classifier may then be used for cross-lingual transfer with low-resource languages.

Systems and methods for cross-lingual transfer in natural language processing

Embodiments described herein provide a semantic parsing framework which may be referred to as Uni-Parser. The Uni-Parser framework may be applied to question answering on both knowledge bases and databases. The three main stages of the Uni-Parser framework are enumeration, ranking, and generation. At the enumeration stage, primitives are enumerated based on matching the question to the data structure. After enumerating primitives, the Uni-Parser framework may rank the primitives used a trained ranker model. The top ranked primitives may then be used as inputs to a generator which is a learned sequence to sequence model which produces a logical form.

Systems and methods for semantic parsing with primitive level enumeration

Embodiments described herein provide a program synthesis framework that generates code programs through a multi-turn conversation between a user and a system. Specifically, the description to solve a target problem is factorized into multiple steps, each of which includes a description in natural language (prompt) to be input into the generation model as a user utterance. The model in turn synthesizes functionally correct subprograms following the current user utterance and considering descriptions and synthesized subprograms at previous steps. The subprograms generated at the multiple steps are then combined to form an output of program in response to the target problem.

Systems and methods for a conversational framework of program synthesis

A system performs group testing on a population of items. The group testing identifies items satisfying particular criteria from a population of items, for example, defective items from the population. The group testing may be performed for software or hardware testing, for testing a human population, for training of deep learning applications, and so on. The system trains a machine learning based model, for example, a reinforcement learning based model to evaluate groups. The model may further determine system dynamics that may represent priors of items. An agent treats the population and groups of items being tested as the environment and performs actions, for example, adjusting the groups. The system also performs a non-adaptive strategy based on monte carlo simulation of tests based on a simulation results.

Reinforcement learning based group testing

Embodiments described herein provide methods and systems for effectively and efficiently summarizing long documents. A transformer is provided with bottom-up and top-down inference combined to effectively capture long-range dependency. In the bottom-up inference, each token only attends to nearby tokens within a window of a specified size. In the top-down inference, full self-attention is given using units with coarser granularity. The bottom-up-inferred token representations are then updated with the top-down representations, which is achieved with cross-attention between the top and token levels. Multiple levels of top-down representations with increasingly coarser granularity can be used if documents are extremely long.

Systems and methods for long document summarization

A conversation engine performs conversations with users using chatbots customized for performing a set of tasks that can be performed using an online system. The conversation engine loads a chatbot configuration that specifies the behavior of a chatbot including the tasks that can be performed by the chatbot, the types of entities relevant to each task, and so on. The conversation may be voice based and use natural language. The conversation engine may load different chatbot configurations to implement different chatbots. The conversation engine receives a conversation engine configuration that specifies the behavior of the conversation engine across chatbots. The system may be a multi-tenant system that allows customization of the chatbots for each tenant.

Growing community of inventors

San Jose, CA, United States of America

Yingbo Zhou