Salesforce, Inc.

Salesforce Com, Inc.

Tong Niu

Caiming Xiong

Yingbo Zhou

Richard Socher

Kazuma Hashimoto

Nitish Shirish Keskar

Nazneen Rajani

Chien-Sheng Wu

Huan Wang

Semih Yavuz

Wenpeng Yin

Xiang Chen

Embodiments described herein adopts a k nearest neighbor (kNN) mechanism over a model's hidden representations to identify training examples closest to a given test example. Specifically, a training set of sequences and a test sequence are received, each of which is mapped to a respective hidden representation vector using a base model. A set of indices for each sequence index that minimizes a distance between the respective hidden state vector and a test hidden state vector is then determined A weighted k-nearest neighbor probability score can then be computed from the set of indices to generate a probability distribution over labels for the test sequence.

Systems and methods for a k-nearest neighbor based mechanism of natural language processing models

Embodiments described herein provide a cross-lingual sentence alignment framework that is trained only on rich-resource language pairs. To obtain an accurate aligner, a pretrained multi-lingual language model is used, and a classifier is trained on parallel data from rich-resource language pairs. This trained classifier may then be used for cross-lingual transfer with low-resource languages.

Systems and methods for cross-lingual transfer in natural language processing

Embodiments described herein provide dynamic blocking, a decoding algorithm which enables large-scale pretrained language models to generate high-quality paraphrases in an un-supervised setting. Specifically, in order to obtain an alternative surface form, when the language model emits a token that is present in the source sequence, the language model is prevented from generating the next token that is the same as the subsequent source token in the source sequence at the next time step. In this way, the language model is forced to generate a paraphrased sequence of the input source sequence, but with mostly different wording.

Systems and methods for unsupervised paraphrase generation

Embodiments described herein provide methods and systems for presenting a document and generating a human-AI summary. A system provides a user with a selection of an amount of time to spend reading the document, or a list of questions from which the user may select which questions they would like answered by reading the document. The system highlights sections of the document according to the user selection. Implicit and explicit user data such as dwell times, user highlights, and user notes, are collected while displaying the document. A human-AI summary is generated based on the document and the user data.

Systems and methods for a collaborative reading assistance tool

Embodiments described herein provide a provide a fully unsupervised model for text compression. Specifically, the unsupervised model is configured to identify an optimal deletion path for each input sequence of texts (e.g., a sentence) and words from the input sequence are gradually deleted along the deletion path. To identify the optimal deletion path, the unsupervised model may adopt a pretrained bidirectional language model (BERT) to score each candidate deletion based on the average perplexity of the resulting sentence and performs a simple greedy look-ahead tree search to select the best deletion for each step.

Growing community of inventors

Sunnyvale, CA, United States of America

Tong Niu