Adobe, Inc.

Nikolaos Barmpalios

Tong Sun

Shawn Alan Gaither

Handong Zhao

Vlad Ion Morariu

Rajiv Jain

Michael F Kraley

Curtis Michael Wigington

Christopher Alan Tensmeyer

Varun Manjunatha

Jiuxiang Gu

Kai Li

Sachin Soni

Franck Dernoncourt

Sanjeev Tagra

Abhishek Jain

Randy L Swineford

Andrew Marc Greene

Shagun Kush

Samarth Gupta

Nargol Rezvani

Naqushab Neyazee

Ruchi Rajiv Deshpande

Jason Wen Yong Kuen

Richard J Cohn

Ani Nenkova

Verena Sabine Kaynig-Fittkau

Sruthi Madapoosi Ravi

Kanchana Sethu

Thi Kim Phung Lai

In implementations of systems for training language models and preserving privacy, a computing device implements a privacy system to predict a next word after a last word in a sequence of words by processing input data using a machine learning model trained on training data to predict next words after last words in sequences of words. The training data describes a corpus of text associated with clients and including sensitive samples and non-sensitive samples. The machine learning model is trained by sampling a client of the clients and using a subset of the sensitive samples associated with the client and a subset of the non-sensitive samples associated with the client to update parameters of the machine learning model. The privacy system generates an indication of the next word after the last word in the sequence of words for display in a user interface.

Training language models and preserving privacy

The technology described includes methods for pretraining a document encoder model based on multimodal self cross-attention. One method includes receiving image data that encodes a set of pretraining documents. A set of sentences is extracted from the image data. A bounding box for each sentence is generated. For each sentence, a set of predicted features is generated by using an encoder machine-learning model. The encoder model performs cross-attention between a set of masked-textual features for the sentence and a set of masked-visual features for the sentence. The set of masked-textual features is based on a masking function and the sentence. The set of masked-visual features is based on the masking function and the corresponding bounding box. A document-encoder model is pretrained based on the set of predicted features for each sentence and pretraining tasks. The pretraining tasks includes masked sentence modeling, visual contrastive learning, or visual-language alignment.

Unified pretraining framework for document understanding

Systems and techniques for privacy preserving document analysis are described that derive insights pertaining to a digital document without communication of the content of the digital document. To do so, the privacy preserving document analysis techniques described herein capture visual or contextual features of the digital document and creates a stamp representation that represents these features without included the content of the digital document. The stamp representation is projected into a stamp embedding space based on a stamp encoding model generated through machine learning techniques capturing feature patterns and interaction in the stamp representations. The stamp encoding model exploits these feature interactions to define similarity of source documents based on location within the stamp embedding space. Accordingly, the techniques described herein can determine a similarity of documents without having access to the documents themselves.

Privacy preserving document analysis

Techniques are disclosed for identifying asides within a document, and detecting a display order of contents based of the identified asides. In a document, an 'aside' represents a content region of the document that is distinct from the main content regions, and may be visually distinguishable from the main content region. In an example, a document is received, where the document lacks identification of asides. The document is analyzed to identify asides within the document. A display order of contents within the document is then determined, based on the identified asides. For example, in the display order, the asides are ordered between two segments of the main content and/or at a beginning or an end of the main content, but may not be ordered to be embedded in between a segment of the main content. The document is displayed in accordance with the display order.

Asides detection in documents

Adapting a machine learning model to process data that differs from training data used to configure the model for a specified objective is described. A domain adaptation system trains the model to process new domain data that differs from a training data domain by using the model to generate a feature representation for the new domain data, which describes different content types included in the new domain data. The domain adaptation system then generates a probability distribution for each discrete region of the new domain data, which describes a likelihood of the region including different content described by the feature representation. The probability distribution is compared to ground truth information for the new domain data to determine a loss function, which is used to refine model parameters. After determining that model outputs achieve a threshold similarity to the ground truth information, the model is output as a domain-agnostic model.

Domain adaptation for machine learning models

Systems, methods, and non-transitory computer-readable media can generate a natural language model that provides user-entity differential privacy. For example, in one or more embodiments, a system samples sensitive data points from a natural language dataset. Using the sampled sensitive data points, the system determines gradient values corresponding to the natural language model. Further, the system generates noise for the natural language model. The system generates parameters for the natural language model using the gradient values and the noise, facilitating simultaneous protection of the users and sensitive entities associated with the natural language dataset. In some implementations, the system generates the natural language model through an iterative process (e.g., by iteratively modifying the parameters).

Preserving user-entity differential privacy in natural language modeling

A domain alignment technique for cross-domain object detection tasks is introduced. During a preliminary pretraining phase, an object detection model is pretrained to detect objects in images associated with a source domain using a source dataset of images associated with the source domain. After completing the pretraining phase, a domain adaptation phase is performed using the source dataset and a target dataset to adapt the pretrained object detection model to detect objects in images associated with the target domain. The domain adaptation phase may involve the use of various domain alignment modules that, for example, perform multi-scale pixel/path alignment based on input feature maps or perform instance-level alignment based on input region proposals.

Domain alignment for object detection domain adaptation tasks

The present disclosure relates to systems, methods, and non-transitory computer readable media for generating diverse and realistic synthetic documents using deep learning. In particular, the disclosed systems can utilize a trained neural network to generate realistic image layouts comprising page elements that comply with layout parameters. The disclosed systems can also generate synthetic content corresponding to the page elements within the image layouts. The disclosed systems insert the synthetic content into the corresponding page elements of documents based on the image layouts to generate synthetic documents.

Automatically generating labeled synthetic documents

Growing community of inventors

Sunnyvale, CA, United States of America

Nikolaos Barmpalios