Salesforce, Inc.

Salesforce Com, Inc.

Wenhao Liu

Caiming Xiong

Tian Xie

Jin Qu

Kazuma Hashimoto

Xinyi Yang

Huan Wang

Yingbo Zhou

Chien-Sheng Wu

Alexis Roos

Feihong Wu

Chu Hong Hoi

Govardana Sachithanandam Ramachandran

Shashank Harinath

Xugang Ye

Zachary Alexander

Chen Xing

Ehsan Hosseini-Asl

Yixin Mao

Ka Chun Au

Richard Socher

Bryan McCann

Nitish Shirish Keskar

Na Cheng

Percy Dara Mehta

Nazneen Rajani

Noah William Burbank

Young Mo Kang

Shardul Vikram

Jesse Vig

Rohit Deshpande

Alexander R Fabbri

Prafulla Kumar Choubey

Wenpeng Yin

Robert Christopher Ames

Linqing Liu

Soujanya Lanka

Joseph Gerald Keller

Victor Winslow Yee

Madhav Vaidyanathan

Madeleine Mary Gill

William Christopher Fama Roller

Alex Michael Noe

Wing Hing Ku

Bhaskar Garg

Jianguo Zhang

Oliver Qian Tang

Shuvajit Das

Ascander Dost

Sammy Adnan Nammari

Joseph R Zeimen

Xu Yang

Janelle Wen Hui Teng

Abraham Dio Suharli

Philippe Laban

Nelson Esteban Acevedo

Liang Qiu

Zahra Fatemi

Sandeep Raju Prabhakar

Embodiments are directed to pre-training a transformer model using more parameters for sophisticated patterns (PSP++). The transformer model is divided into a held-out model and a main model. A forward pass and a backward pass are performed on the held-out model, where the forward pass determines self-attention hidden states of the held-out model and the backward pass determines loss of the held-out model. A forward pass on the main model is performed to determine a self-attention hidden states of the main model. The self-attention hidden states of the main model are concatenated with the self-attention hidden states of the held-out model. A backward pass is performed on the main model to determine a loss of the main model. The parameters of the held-out model are updated to reflect the loss of the held-out model and parameters of the main model are updated to reflect the loss of the main model.

Parameter utilization for language pre-training

Embodiments described herein provide a document summarization framework that employs an ensemble of summarization models, each of which is a modified version of a base summarization model to control hallucination. For example, a base summarization model may first be trained on a full training data set. The trained base summarization model is then fine-tuned using a first filtered subset of the training data which contains noisy data, resulting in an 'anti-expert' model. The parameters of the anti-expert model are subtracted from the parameters of the trained base model to produce a final summarization model which yields robust factual performance.

Systems and methods for parameter ensembling for reducing hallucination in abstractive summarization

Some embodiments of the current disclosure disclose methods and systems for training for training a natural language processing intent classification model to perform few-shot classification tasks. In some embodiments, a pair of an utterance and a first semantic label labeling the utterance may be generated and a neural network that is configured to perform natural language inference tasks may be utilized to determine the existence of an entailment relationship between the utterance and the semantic label. The semantic label may be predicted as the intent class of the utterance based on the entailment relationship and the pair may be used to train the natural language processing intent classification model to perform few-shot classification tasks.

Systems and methods for few-shot intent classifier models

Sentiment analysis is a task in natural language processing. The embodiments are directed to using a generative language model to extract an aspect term, aspect category and their corresponding polarities. The generative language model may be trained as a single, joint, and multi-task model. The single-task generative language model determines a term polarity from the aspect term in the sentence or a category polarity from an aspect category in the sentence. The joint-task generative language model determines both the aspect term and the term polarity or the aspect category and the category polarity. The multi-task generative language model determines the aspect term, term polarity, aspect category and category polarity of the sentence.

Generative language model for few-shot aspect-based sentiment analysis

Dialogue summarization is challenging due to its multi-speaker standpoints, casual spoken language style, and limited labelled data. The embodiments are directed to a coarse-to-fine dialogue summarization model that improves abstractive dialogue summarization quality and enables granular controllability. A summary draft that includes key words for turns in a dialogue conversation history is created. The summary draft includes pseudo-labelled interrogative pronoun categories and noisy key phrases. The dialogue conversation history is divided into segments. A generate language model is trained to generate a segment summary for each dialogue segment using a portion of the summary draft that corresponds to at least one dialogue turn in the dialogue segment. A dialogue summary is generated using the generative language model trained using the summary draft.

Coarse-to-fine abstractive dialogue summarization with controllable granularity

Database systems and methods are provided for assigning structural metadata to records and creating automations using the structural metadata. One method of assigning structural metadata to a record associated with a conversation involves obtaining a plurality of utterances associated with the conversation, identifying, from among the plurality of utterances, a representative utterance for semantic content of the conversation, assigning the conversation to a group of semantically similar conversations based on the representative utterance, and automatically updating the record associated with the conversation at a database system to include metadata identifying the group of semantically similar conversations.

Database systems with automated structural metadata assignment

Embodiments described herein provide a method of evaluating a natural language processing model. The method includes receiving an evaluation dataset that may include a plurality of unit tests, the unit tests having: an input context, and a first candidate and a second candidate that are generated in response to the input context, where the first test candidate is associated with a first quality notation, and the second candidate is associated with a second quality notation. The method includes determining, via a model, a first likelihood of generating the first candidate and a second likelihood of generating the second candidate in response to the input context. The method also includes determining whether the first likelihood being greater than the second likelihood. The method also includes determining whether the first model passed the unit test, where the first quality notation indicates a higher quality candidate and the second quality notation indicate a lower quality candidate.

Systems and methods near negative distinction for evaluating NLP models

Embodiments described herein propose an approach for unsupervised structure extraction in task-oriented dialogues. Specifically, a Slot Boundary Detection (SBD) module is adopted, for which utterances from training domains are tagged with the conventional BIO schema but without the slot names. A transformer-based classifier is trained to detect the boundary of potential slot tokens in the test domain. Next, while the state number is usually unknown, it is more reasonable to assume the slot number is given when analyzing a dialogue system. The detected tokens are clustered into the number of slot of groups. Finally, the dialogue state is represented with a vector recording the modification times of every slot. The slot values are then tracked through each dialogue session in the corpus and label utterances with their dialogue states accordingly. The semantic structure is portrayed by computing the transition frequencies among the unique states.

Systems and methods for unsupervised structure extraction in task-oriented dialogues

Embodiments are directed to a training framework for reducing gender bias in a pre-trained language model. To reduce gender bias a gender neutral dataset is generated. Next, parameters of the pre-trained language model are frozen and do not change during a subsequent training phase. As all the pre-trained parameters are frozen, forgetting of information from the original training data is minimized. New parameters are added to the language model. The new parameters may be associated with gender related terms, such as profession names. In a subsequent training phase the new parameters of the language model are trained using a gender neutral dataset.

Systems and methods for refining pre-trained language models with improved gender fairness

Embodiments described herein provides a training mechanism that transfers the knowledge from a trained BERT model into a much smaller model to approximate the behavior of BERT. Specifically, the BERT model may be treated as a teacher model, and a much smaller student model may be trained using the same inputs to the teacher model and the output from the teacher model. In this way, the student model can be trained within a much shorter time than the BERT teacher model, but with comparable performance with BERT.

Systems and methods for distilled BERT-based training model for text classification

A conversation engine performs conversations with users using chatbots customized for performing a set of tasks that can be performed using an online system. The conversation engine loads a chatbot configuration that specifies the behavior of a chatbot including the tasks that can be performed by the chatbot, the types of entities relevant to each task, and so on. The conversation may be voice based and use natural language. The conversation engine may load different chatbot configurations to implement different chatbots. The conversation engine receives a conversation engine configuration that specifies the behavior of the conversation engine across chatbots. The system may be a multi-tenant system that allows customization of the chatbots for each tenant.

Customizing chatbots based on user specification

Configurable conversation engine for executing customizable chatbots

A system uses conversation engines to process natural language requests and conduct automatic conversations with users. The system generates responses to users in an online conversation. The system ranks generated user responses for the online conversation. The system generates a context vector based on a sequence of utterances of the conversation and generates response vectors for generated user responses. The system ranks the user responses based on a comparison of the context vectors and user response vectors. The system uses a machine learning based model that uses a pretrained neural network that supports multiple languages. The system determines a context of an utterance based on utterances in the conversation. The system generates responses and ranks them based on the context. The ranked responses are used to respond to the user.

Machine learning based models for automatic conversations in online systems

Growing community of inventors

Redwood City, CA, United States of America

Wenhao Liu