Adobe, Inc.

Puneet Mangla

Balaji Krishnamurthy

Nupur Kumari

Mayank Kumar Singh

Milan Aggarwal

Abhishek Sinha

Arneh Jain

Salil Taneja

Gaurav Ahuja

Various embodiments classify one or more portions of an image based on deriving an 'intrinsic' modality. Such intrinsic modality acts as a substitute to a “text” modality in a multi-modal network. A text modality in image processing is typically a natural language text that describes one or more portions of an image. However, explicit natural language text may not be available across one or more domains for training a multi-modal network. Accordingly, various embodiments described herein generate an intrinsic modality, which is also a description of one or more portions of an image, except that such description is not an explicit natural language description, but rather a machine learning model representation. Some embodiments additionally leverage a visual modality obtained from a vision-only model or branch, which may learn domain characteristics that are not present in the multi-modal network. Some embodiments additionally fuse or integrate the intrinsic modality with the visual modality for better generalization.

Using intrinsic multimodal features of image for domain generalized

In implementations of systems for assistive digital form authoring, a computing device implements an authoring system to receive input data describing a search input associated with a digital form. The authoring system generates an input embedding vector that represents the search input in a latent space using a machine learning model trained on training data to generate embedding vectors in the latent space. A candidate embedding vector included in a group of candidate embedding vectors is identified based on a distance between the input embedding vector and the candidate embedding vector in the latent space. The authoring system generates an indication of a search output associated with the digital form for display in a user interface based on the candidate embedding vector.

Assistive digital form authoring

Embodiments are disclosed for training a neural network classifier to learn to more closely align an input image with its attribution map. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving a training image comprising a representation of one or more objects, the training image associated with at least one label for the representation of the one or more objects, generating a perturbed training image based on the training image using a neural network, and training the neural network using the perturbed training image by minimizing a combination of classification loss and attribution loss to learn to align an image with its corresponding attribution map.

Attributionally robust training for weakly supervised localization and segmentation

The present disclosure relates to systems, methods, and non-transitory computer readable media for training a classification neural network to classify digital images in few-shot tasks based on self-supervision and manifold mixup. For example, the disclosed systems can train a feature extractor as part of a base neural network utilizing self-supervision and manifold mixup. Indeed, the disclosed systems can apply manifold mixup regularization over a feature manifold learned via self-supervised training such as rotation training or exemplar training. Based on training the feature extractor, the disclosed systems can also train a classifier to classify digital images into novel classes not present within the base classes used to train the feature extractor.

Growing community of inventors

Faridabad, India

Puneet Mangla