Adobe, Inc.

Sheng Li

Handong Zhao

Zhe Lin

Sungchul Kim

Mingyang Ling

Jiuxiang Gu

Viswanathan Swaminathan

Zhaowen Wang

Trung Huu Bui

Chen Fang

Saayan Mitra

Branislav Kveton

Hung Hai Bui

Somdeb Sarkhel

Zheng Wen

Jaya Kawale

Tuan Manh Lai

Xiaowei Jia

Xueyu Mao

Zhiqiang Tao

Quan Hung Tran

Methods, systems, and non-transitory computer readable storage media are disclosed for generating semantic scene graphs for digital images using an external knowledgebase for feature refinement. For example, the disclosed system can determine object proposals and subgraph proposals for a digital image to indicate candidate relationships between objects in the digital image. The disclosed system can then extract relationships from an external knowledgebase for refining features of the object proposals and the subgraph proposals. Additionally, the disclosed system can generate a semantic scene graph for the digital image based on the refined features of the object/subgraph proposals. Furthermore, the disclosed system can update/train a semantic scene graph generation network based on the generated semantic scene graph. The disclosed system can also reconstruct the image using object labels based on the refined features to further update/train the semantic scene graph generation network.

Generating scene graphs from digital images using external knowledge and image reconstruction

Systems and techniques for multi-task equidistant embedding are described that process categorical feature data to explore feature interactions. A digital analytics system enforces an equidistant relationship among features within a category while extracting high-order feature interactions by punishing both positive correlations and negative correlations among low-dimensional representations of different features. By enforcing an equidistant embedding, information is retained and accuracy is increased while higher order feature interactions are determined. Further, the digital analytics system shares knowledge among different tasks by connecting a shared network representation common to multiple tasks with exclusive network representations specific to particular tasks.

Multi-task equidistant embedding

The present description relates to systems, methods, and non-transitory computer readable media for generating digital responses to digital queries by utilizing a classification model and query-specific analysis models. For example, the described systems can train a classification model to generate query classifications corresponding to product queries, conversational queries, and/or recommendation/purchase queries. Moreover, the described systems can apply the classification model to select pertinent models for particular queries. For example, upon classifying a product query, the described systems can utilize a neural ranking model (trained based on a set of training product specifications and training queries) to generate relevance scores for product specifications associated with a digital query. The described systems can further compare generated relevance scores to select a product specification and generate a digital response that includes the pertinent product specification to provide for display to a client device.

Generating and utilizing classification and query-specific models to generate digital responses to queries from client device

Techniques are disclosed for the generation of adversarial training data through sequence perturbation, for a deep learning network to perform event sequence analysis. A methodology implementing the techniques according to an embodiment includes applying a long short-term memory attention model to an input data sequence to generate discriminative sequence periods and attention weights associated with the discriminative sequence periods. The attention weights are generated to indicate the relative importance of data in those discriminative sequence periods. The method further includes generating perturbed data sequences based on the discriminative sequence periods and the attention weights. The generation of the perturbed data sequences employs selective filtering or conservative adversarial training, to preserve perceptual similarity between the input data sequence and the perturbed data sequences. The input data sequence may be created by vectorizing a temporal input data stream comprising words, symbols, and the like, into a multidimensional vectorized numerical data sequence format.

Adversarial training for event sequence analysis

Methods and systems are provided for generating interpretable user modeling system. The interpretable user modeling system can use an intent neural network to implement one or more tasks. The intent neural network can bridge a semantic gap between log data and human language by leveraging tutorial data to understand user logs in a semantically meaningful way. A memory unit of the intent neural network can capture information from the tutorial data. Such a memory unit can be queried to identify human readable sentences related to actions received by the intent neural network. The human readable sentences can be used to interpret the user log data in a semantically meaningful way.

Interpretable user modeling from unstructured user data

Techniques are disclosed for training of factorization machines (FMs) using a streaming mode alternating least squares (ALS) optimization. A methodology implementing the techniques according to an embodiment includes receiving a datapoint that includes a feature vector and an associated target value. The feature vector includes user identification, subject matter identification, and a context. The target value identifies an opinion of the user relative to the subject matter. The method further includes applying an FM to the feature vector to generate an estimate of the target value, and updating parameters of the FM for training of the FM. The parameter update is based on application of a streaming mode ALS optimization to: the datapoint; the estimate of the target value; and to an updated summation of intermediate calculated terms generated by application of the streaming mode ALS optimization to previously received datapoints associated with prior parameter updates of the FM.

Online training and update of factorization machines using alternating least squares optimization

A deep collaborative filtering (DCF) approach is employed in a recommender system to provide item recommendations to users. The DCF approach combines deep learning models with matrix factorization based collaborative filtering. To provide item recommendations, a user-item rating matrix, user side information, and item side information are provided as input to a recommender system. The recommender system learns user latent factors and item latent factors by jointly: (1) decomposing the user-item rating matrix to extract latent factors, and (2) extracting latent factors from hidden layers of deep learning models using the user side information and item side information. The learned user latent factors and item latent factors are used to predict item ratings for missing ratings in the user-item rating matrix. The predicted item ratings are then used to select item recommendations for a given user, which are then communicated to a user device of the user.

Item recommendations via deep collaborative filtering

Growing community of inventors

Suwanee, GA, United States of America

Sheng Li