Adobe, Inc.

Varun Manjunatha

Vlad Ion Morariu

Rajiv Jain

Franck Dernoncourt

Handong Zhao

Tong Sun

Curtis Michael Wigington

Christopher Alan Tensmeyer

Nedim Lipka

Nikolaos Barmpalios

Kai Li

Joseph Barrow

Walter Chang

Sasha Spala

Nicholas Miller

Hailin Jin

Jonathan Worthen Brandt

Trung Huu Bui

Quan Anh Tran

Doo Soon Kim

Amir Pouran Ben Veyseh

Jiuxiang Gu

Ashutosh Mehra

Lidan Wang

Ani Nenkova

Timothy Jeewun Ganter

Sangwoo Cho

Jason Kuen

Peizhao Li

Pramuditha Perera

Vitali Petsiuk

Vicente Ignacio Ordonez Roman

Danilo Neves Ribeiro

Certain embodiments involve using a machine-learning tool to generate metadata identifying segments and topics for text within a document. For instance, in some embodiments, a text processing system obtains input text and applies a segmentation-and-labeling model to the input text. The segmentation-and-labeling model is trained to generate a predicted segment for the input text using a segmentation network. The segmentation-and-labeling model is also trained to generate a topic for the predicted segment using a pooling network of the model to the predicted segment. The output of the model is usable for generating metadata identifying the predicted segment and the associated topic.

Machine-learning tool for generating segmentation and topic metadata for documents

System and methods for a text summarization system are described. In one example, a text summarization system receives an input utterance and determines whether the utterance should be included in a summary of the text. The text summarization system includes an embedding network, a convolution network, an encoding component, and a summary component. The embedding network generates a semantic embedding of an utterance. The convolution network generates a plurality of feature vectors based on the semantic embedding. The encoding component identifies a plurality of latent codes respectively corresponding to the plurality of feature vectors. The summary component identifies a prominent code among the latent codes and to select the utterance as a summary utterance based on the prominent code.

Topical vector-quantized variational autoencoders for extractive summarization of video transcripts

Systems and methods for natural language processing are described. One or more embodiments of the present disclosure identify a claim from a document, wherein the claim corresponds to a topic, create a graph comprising a plurality of nodes having a plurality of node types and a plurality of edges having a plurality of edge types, wherein one of the nodes represents the claim, and wherein each of the edges represents a relationship between a corresponding pair of the nodes, encode the claim based on the graph using a graph convolutional network (GCN) to obtain an encoded claim, classify the claim by decoding the encoded claim to obtain a stance label that indicates a stance of the claim towards the topic, and transmit information indicating a viewpoint of the document towards the topic based on the stance label.

Syntopical reading for collection understanding

Adapting a machine learning model to process data that differs from training data used to configure the model for a specified objective is described. A domain adaptation system trains the model to process new domain data that differs from a training data domain by using the model to generate a feature representation for the new domain data, which describes different content types included in the new domain data. The domain adaptation system then generates a probability distribution for each discrete region of the new domain data, which describes a likelihood of the region including different content described by the feature representation. The probability distribution is compared to ground truth information for the new domain data to determine a loss function, which is used to refine model parameters. After determining that model outputs achieve a threshold similarity to the ground truth information, the model is output as a domain-agnostic model.

Domain adaptation for machine learning models

Systems and methods for natural language processing are described. One or more embodiments of the present disclosure receive a document comprising a plurality of words organized into a plurality of sentences, the words comprising an event trigger word and an argument candidate word, generate word representation vectors for the words, generate a plurality of document structures including a semantic structure for the document based on the word representation vectors, a syntax structure representing dependency relationships between the words, and a discourse structure representing discourse information of the document based on the plurality of sentences, generate a relationship representation vector based on the document structures, and predict a relationship between the event trigger word and the argument candidate word based on the relationship representation vector.

Inducing rich interaction structures between words for document-level event argument extraction

One example method involves operations for a processing device that include receiving, by a machine learning model trained to generate a search result, a search query for a text input. The machine learning model is trained by receiving pre-training data that includes multiple documents. Pre-training the machine learning model by generating, using an encoder, feature embeddings for each of the documents included in the pre-training data. The feature embeddings are generated by applying a masking function to visual and textual features in the documents. Training the machine learning model also includes generating, using the feature embeddings, output features for the documents by concatenating the feature embeddings and applying a non-linear mapping to the feature embeddings. Training the machine learning model further includes applying a linear classifier to the output features. Additionally, operations include generating, for display, a search result using the machine learning model based on the input.

Self-supervised document representation learning

Embodiments are disclosed for performing fact correction of natural language sentences using data tables. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving an input sentence, tokenizing elements of the input sentence, and identifying, by a first machine learning model, a data table associated with the input sentence. The systems and methods further comprise a second machine learning model identifying a tokenized element of the input sentence that renders the input sentence false based on the data table and masking the tokenized element of the tokenized input sentence that renders the input sentence false. The systems and method further includes a third machine learning model predicting a new value for the masked tokenized element based on the input sentence with the masked tokenized element and the identified data table and providing an output including a modified input sentence with the new value.

Fact correction of natural language sentences using data tables

The present disclosure relates to systems, non-transitory computer-readable media, and methods for classifying an input image utilizing a classification model conditioned by a generative model and/or self-supervision. For example, the disclosed systems can utilize a generative model to generate a reconstructed image from an input image to be classified. In turn, the disclosed systems can combine the reconstructed image with the input image itself. Using the combination of the input image and the reconstructed image, the disclosed systems utilize a classification model to determine a classification for the input image. Furthermore, the disclosed systems can employ self-supervised learning to cause the classification model to learn discriminative features for better classifying images of both known classes and open-set categories.

Classifying images utilizing generative-discriminative feature representations

A domain alignment technique for cross-domain object detection tasks is introduced. During a preliminary pretraining phase, an object detection model is pretrained to detect objects in images associated with a source domain using a source dataset of images associated with the source domain. After completing the pretraining phase, a domain adaptation phase is performed using the source dataset and a target dataset to adapt the pretrained object detection model to detect objects in images associated with the target domain. The domain adaptation phase may involve the use of various domain alignment modules that, for example, perform multi-scale pixel/path alignment based on input feature maps or perform instance-level alignment based on input region proposals.

Domain alignment for object detection domain adaptation tasks

Introduced here are computer programs and associated computer-implemented techniques for creating visualizations to explain the outputs produced by models designed for object detection. To accomplish this, a graphics editing platform can obtain a reference output that identifies a region of pixels in a digital image that allegedly contains an object. Then, the graphics editing platform can compute the similarity between the reference output and a series of outputs generated by a model upon being applied to masked versions of the digital image. A visualization component can be produced based on the similarity.

Explanatory visualizations for object detection

Growing community of inventors

College Park, MD, United States of America

Varun Manjunatha