Adobe, Inc.

Ebay, Incorporated

Ajinkya Gorakhnath Kale

Robinson Piramuthu

Mohammadhadi Kiapour

Pranav Vineet Aggarwal

Qiaosong Wang

Fan Yang

Zhe Lin

Baldo Antonio Faieta

Amit Srivastava

Midhun Harikumar

Licheng Yu

Ritiz Tambi

Trung Huu Bui

Venkata Naveen Kumar Yadav Marri

Stefan Schoenmackers

Handong Zhao

Sanjika Hewavitharana

Manojkumar Rangasamy Kannadasan

Zhaowen Wang

Franck Dernoncourt

Saeid Motiian

Tracy Holloway King

Thrivikrama Taula

Dishan Gupta

Zhankui He

Sachin Soni

Carl Dockhorn

William F Kraus

Ali Aminian

Lawrence William Colagiovanni

Aashish Misraa

Sasha Spala

Nicholas Miller

Yu Gong

Fengbin Chen

Kate Sousa

Alvin Ghouas

Benjamin Leviant

Venkat Barakam

Nathaniel Joseph Grabaskas

Joon Kim

Tushar Dublish

Francois Guerin

Vinh Khuc

Jianming Zhang

Yilin Wang

Shabnam Ghadar

Ryan Rozich

Michele Saad

Sanyam Jain

Jason Wen Yong Kuen

Oliver Brdiczka

Taesung Park

Alex Charles Filipkowski

Peter Evan O'Donovan

Sourabh Goel

Seunghyun Yoon

Abhishek Gupta

Vinh Ngoc Khuc

Xiaowei Jia

Roland Geisler

Piyush Chandra

Nitin Kumar Garg

Sachin Madhav Kelkar

Xin Yuan

Rishabh Purwar

Sukriti Verma

Dylan Nathaniel Warnock

Mikhail Kotov

Hareesh Ravi

Naveen Marri

Jaemin Cho

Rishav Agarwal

Di Pu

Daniel ReMine

Vikas Sagar

Deepika Naryani

Feroz Ahmad

Systems and methods for image processing, and specifically for generating object-agnostic image representations, are described. Embodiments of the present disclosure receive a training image including a foreground object and a background, remove the foreground object from the training image to obtain a modified training image, inpaint a portion of the modified training image corresponding to the foreground object to obtain an inpainted training image, encode the training image and the inpainted training image using a machine learning model to obtain an encoded training image and an encoded inpainted training image, and update parameters of the machine learning model based on the encoded training image and the encoded inpainted training image.

Object-agnostic image representation

Systems and methods for image processing are described. Embodiments of the present disclosure obtain a text prompt for text guided image generation. A multi-modal encoder of an image processing apparatus encodes the text prompt to obtain a text embedding. A diffusion prior model of the image processing apparatus converts the text embedding to an image embedding. A latent diffusion model of the image processing apparatus generates an image based on the image embedding, wherein the image includes an element described by the text prompt.

Prior guided latent diffusion

Example embodiments that analyze images to characterize aspects of the images rely on a same neural network to characterize multiple aspects in parallel. Because additional neural networks are not required for additional aspects, such an approach scales with increased aspects.

Parallel prediction of multiple image aspects

Systems and methods for image processing are described. Embodiments of the present disclosure include an image generation network configured to encode a plurality of abstract images using a style encoder to obtain a plurality of abstract style encodings, wherein the style encoder is trained to represent image style separately from image content. A clustering component clusters the plurality of abstract style encodings to obtain an abstract style cluster comprising a subset of the plurality of abstract style encodings. A preset component generates an abstract style transfer preset representing the abstract style cluster.

Preset style transfer

A method and system for color optimization in generated images are described. The method and system include receiving an image generation prompt that includes a text description of target image content and color information describing a target color palette; encoding the image generation prompt to obtain image features that represent the target image content and the target color palette; and generating an image representing the target image content with the target color palette based on the image features.

Systems and methods for color palette optimization

Systems and methods for query processing are described. Embodiments of the present disclosure identify a target phrase in an original query, wherein the target phrase comprises a phrase to be replaced in the original query; replace the target phrase with a mask token to obtain a modified query; generate an alternative query based on the modified query using a masked language model (MLM), wherein the alternative query includes an alternative phrase in place of the target phrase that is consistent with a context of the target phrase; and retrieve a search result based on the alternative query.

Zero-shot entity-aware nearest neighbors retrieval

Techniques for generating a novel image using tokenized image representations are disclosed. In some embodiments, a method of generating the novel image includes generating, via a first machine learning model, a first sequence of coded representations of a first image having one or more features; generating, via a second machine learning model, a second sequence of coded representations of a sketch image having one or more edge features associated with the one or more features; predicting, via a third machine learning model, one or more subsequent coded representations based on the first sequence of coded representations and the second sequence of coded representations; and based on the subsequent coded representations, generating, via the third machine learning model, a first portion of a reconstructed image having one or more image attributes of the first image, and a second portion of the reconstructed image associated with the one or more edge features.

Generating novel images using sketch image representations

Technology is disclosed herein for enhanced similarity search. In an implementation, a search environment includes one or more computing hardware, software, and/or firmware components in support of enhanced similarity search. The one or more components identify a modality for a similarity search with respect to a query object. The components generate an embedding for the query object based on the modality and based on connections between the query object and neighboring nodes in a graph. The embedding for the query object provides the basis for the search for similar objects.

Unified framework for multi-modal similarity search

Systems and methods for image processing are configured. Embodiments of the present disclosure identify target style attributes and target structure attributes for a composite image; generate a matrix of composite feature tokens based on the target style attributes and the target structure attributes, wherein subsequent feature tokens of the matrix of composite feature tokens are sequentially generated based on previous feature tokens of the matrix of composite feature tokens according to a linear ordering of the matrix of composite feature tokens; and generate the composite image based on the matrix of composite feature tokens, wherein the composite image includes the target style attributes and the target structure attributes.

Unsupervised style and color cues for transformer-based image generation

Various embodiments use a neural network to analyze images for aspects that characterize the images, to present locations of those aspects on the images, and, additionally, to permit a user to interact with those locations on the images. For example, a user may interact with a visual cue over one of those locations to modify, refine, or filter the results of a visual search, performed on a publication corpus, that uses an input image (e.g., one captured using a mobile device) as a search query.

Visual aspect localization presentation

Systems, methods, and computer program products for identifying a candidate product in an electronic marketplace based on a visual comparison between candidate product image visual content and input query image visual content. Embodiments generate and store descriptive image signatures from candidate product images or selected portions of such images. A subsequently calculated visual similarity measure serves as a visual search result score for the candidate product in comparison to an input query image. Any number of images of any number of candidate products may be analyzed, such as for items available for sale in an online marketplace. Image analysis results are stored in a database and made available for subsequent automated on-demand visual comparisons to an input query image. The embodiments enable substantially real time visual based product searching of a potentially vast catalog of items.

Intelligent online personal assistant with offline visual search database

Embodiments are disclosed for machine learning-based generation of recommended layouts. The method includes receiving a set of design elements for performing generative layout recommendation. A number of each type of design element from the set of design elements is determined. A set of recommended layouts are generated using a trained generative layout model and the number and type of design elements. The set of recommended layouts are output.

Machine learning-based layout generation

Systems and methods for image processing are described. Embodiments of the present disclosure identify a plurality of candidate concepts in a knowledge graph (KG) that correspond to an image tag of an image; generate an image embedding of the image using a multi-modal encoder; generate a concept embedding for each of the plurality of candidate concepts using the multi-modal encoder; select a matching concept from the plurality of candidate concepts based on the image embedding and the concept embedding; and generate association data between the image and the matching concept.

Concept disambiguation using multimodal embeddings

Systems and methods for multi-modal representation learning are described. One or more embodiments provide a visual representation learning system trained using machine learning techniques. For example, some embodiments of the visual representation learning system are trained using cross-modal training tasks including a combination of intra-modal and inter-modal similarity preservation objectives. In some examples, the training tasks are based on contrastive learning techniques.

Visual-semantic representation learning via multi-modal contrastive training

Systems and methods for image captioning are described. One or more aspects of the systems and methods include generating a training caption for a training image using an image captioning network; encoding the training caption using a multi-modal encoder to obtain an encoded training caption; encoding the training image using the multi-modal encoder to obtain an encoded training image; computing a reward function based on the encoded training caption and the encoded training image; and updating parameters of the image captioning network based on the reward function.

Image captioning

Growing community of inventors

San Jose, CA, United States of America

Ajinkya Gorakhnath Kale