Adobe, Inc.

Quan Hung Tran

Trung Huu Bui

Walter Chang

Franck Dernoncourt

Zhe Lin

Seunghyun Yoon

Kushal Kafle

Scott David Cohen

Nedim Lipka

Zhihong Ding

Rajiv Jain

Vlad Ion Morariu

Jiuxiang Gu

Xiang Chen

Long Mai

Ashutosh Mehra

Anandhavelu N

Khoi Pham

Ani Nenkova

Zhuowan Li

Puneet Mathur

Adyasha Maharana

Jianguo Zhang

Xuanli He

Qiuyu Chen

Verena Kaynig-Fittkau

A computer-implemented method is disclosed for determining one or more characteristics of a dialog between a computer system and user. The method may comprise receiving a system utterance comprising one or more tokens defining one or more words generated by the computer system; receiving a user utterance comprising one or more tokens defining one or more words uttered by a user in response to the system utterance, the system utterance and the user utterance forming a dialog context; receiving one or more utterance candidates comprising one or more tokens; for each utterance candidate, generating an input sequence combining the one or more tokens of each of the system utterance, the user utterance, and the utterance candidate; and for each utterance candidate, evaluating the generated input sequence with a model to determine a probability that the utterance candidate is relevant to the dialog context.

Methods and systems for determining characteristics of a dialog between a computer and a user

Systems and methods for color prediction are described. Embodiments of the present disclosure receive an image that includes an object including a color, generate a color vector based on the image using a color classification network, where the color vector includes a color value corresponding to each of a set of colors, generate a bias vector by comparing the color vector to teach of a set of center vectors, where each of the set of center vectors corresponds to a color of the set of colors, and generate an unbiased color vector based on the color vector and the bias vector, where the unbiased color vector indicates the color of the object.

Systems and methods for predicting long-tail colors in images

Systems and methods for natural language processing are described. Embodiments are configured to receive a structured representation of a search query, wherein the structured representation comprises a plurality of nodes and at least one edge connecting two of the nodes, receive a modification expression for the search query, wherein the modification expression comprises a natural language expression, generate a modified structured representation based on the structured representation and the modification expression using a neural network configured to combine structured representation features and natural language expression features, and perform a search based on the modified structured representation.

Scene graph modification based on natural language commands

The present disclosure relates to systems, methods, and non-transitory computer-readable media that generate a digital document hierarchy comprising layers of parent-child element relationships from the visual elements. For example, for a layer of the layers, the disclosed systems determine, from the visual elements, candidate parent visual elements and child visual elements. In addition, for the layer of the layers, the disclosed systems generate, from the feature embeddings utilizing a neural network, element classifications for the candidate parent visual elements and parent-child element link probabilities for the candidate parent visual elements and the child visual elements. Moreover, for the layer, the disclosed systems select parent visual elements from the candidate parent visual elements based on the parent-child element link probabilities. Further, the disclosed systems utilize the digital document hierarchy to generate an interactive digital document from the digital document image.

Extracting document hierarchy using a multimodal, layer-wise link prediction neural network

Systems and methods for natural language processing are described. One or more aspects of a method, apparatus, and non-transitory computer readable medium include receiving a text phrase; encoding the text phrase using an encoder to obtain a hidden representation of the text phrase, wherein the encoder is trained during a first training phrase using self-supervised learning based on a first contrastive loss and during a second training phrase using supervised learning based on a second contrastive learning loss; identifying an intent of the text phrase from a predetermined set of intent labels using a classification network, wherein the classification network is jointly trained with the encoder in the second training phase; and generating a response to the text phrase based on the intent.

Intent detection

This disclosure describes one or more implementations of systems, non-transitory computer-readable media, and methods that extract multiple attributes from an object portrayed in a digital image utilizing a multi-attribute contrastive classification neural network. For example, the disclosed systems utilize a multi-attribute contrastive classification neural network that includes an embedding neural network, a localizer neural network, a multi-attention neural network, and a classifier neural network. In some cases, the disclosed systems train the multi-attribute contrastive classification neural network utilizing a multi-attribute, supervised-contrastive loss. In some embodiments, the disclosed systems generate negative attribute training labels for labeled digital images utilizing positive attribute labels that correspond to the labeled digital images.

Extracting attributes from arbitrary digital images utilizing a multi-attribute contrastive classification neural network

Systems and methods for intent discovery and video summarization are described. Embodiments of the present disclosure receive a video and a transcript of the video, encode the video to obtain a sequence of video encodings, encode the transcript to obtain a sequence of text encodings, apply a visual gate to the sequence of text encodings based on the sequence of video encodings to obtain gated text encodings, and generate an intent label for the transcript based on the gated text encodings.

Multimodal intent discovery system

A group captioning system includes computing hardware, software, and/or firmware components in support of the enhanced group captioning contemplated herein. In operation, the system generates a target embedding for a group of target images, as well as a reference embedding for a group of reference images. The system identifies information in-common between the group of target images and the group of reference images and removes the joint information from the target embedding and the reference embedding. The result is a contrastive group embedding that includes a contrastive target embedding and a contrastive reference embedding with which to construct a contrastive group embedding, which is then input to a model to obtain a group caption for the target group of images.

Contrastive captioning for image groups

Growing community of inventors

San Jose, CA, United States of America

Quan Hung Tran