Amazon Technologies, Inc.

Microsoft Technology Licensing, LLC

Amit Adam

Shai Mazor

Rahul Bhotika

Hadar Averbuch Elor

Gerard Guy Medioni

Ehud Benyamin Rivlin

Stefano Soatto

Sebastian Nowozin

Oron Anschel

Andrea Olgiati

Igor Kviatkovsky

Gal Sabina Star

Or Perel

Wendy Tse

Omer Yair

Shahar Tsiper

Bhavesh Anil Doshi

Vijay Mahadevan

Erez Tadmor

Anand Dhandhania

Umar Farooq

Kunwar Yashraj Singh

Patrick Ian Wilson

Roee Litman

Mohammed El Hamalawi

Simon M Guest

Omri Ben-Eliezer

Gururaj Kosuru

Michael Schober

Christoph Dann

Techniques for cryptographically protecting personally identifiable information in images and videos are described herein. An image may be obtained. One or more regions in the image may be detected based on an object detection algorithm. Pixels for each region of the one or more regions may be encrypted using a symmetric encryption technique and an initialization vector. The encrypted pixels for each region of the one or more regions may be written back into the image. A symmetric key of the symmetric encryption technique and the initialization vector may be encrypted using an asymmetric encryption technique. Metadata of the image may be updated to store the encrypted symmetric key and the encrypted initialization vector.

Cryptographic protection for portions of media

Described is a method for processing image data to determine if a portion of the imaged environment is exposed to high illumination, such as sunlight. In some implementations, image data from multiple different imaging devices may be processed to produce for each imaging device a respective illumination mask that identifies pixels that represent a portion of the environment that is exposed to high illumination. Overlapping portions of those illumination masks may then be combined to produce a unified illumination map of an area of the environment. The unified illumination map identifies, for different portions of the environment, a probability that the portion is actually exposed to high illumination.

Image data and simulated illumination maps

Techniques for document rectification via homography recovery using machine learning are described. An image rectification system can intelligently make use of multiple pipelines for rectifying document images based on the detected type of device that generated the images. The image rectification system can provide high-quality rectifications without requiring human cooperation, multiple views of the document in multiple images, and/or without being constrained to only be able to process images from one source context.

Document rectification via homography recovery using machine learning

Techniques for auto-generation of annotated real-world training data are described. An electronic document is analyzed to determine text represented in the document and corresponding locations of the text. A representation of the electronic document is modified to include markers and printed. The printed document is photographed in real-world environments, and the markers within the digital photographs are analyzed to allow for the depiction of the document within the photographs to be rectified. The text and location data are used to annotate the rectified images.

Auto-annotation techniques for text localization

Techniques for grammar-based automated generation of annotated synthetic form training data for machine learning are described. A training data generation engine utilizes a defined grammar to construct a layout for a form, select key-value units to place within the layout, and select attribute variants for the key-value units. The form is rendered and stored at a storage location, where it can be provided along with other similarly-generated forms to be used as training data for a machine learning model.

Grammar-based automated generation of annotated synthetic form training data for machine learning

Techniques for layout-agnostic complex document processing are described. A document processing service can analyze documents that do not adhere to defined layout rules in an automated manner to determine the content and meaning of a variety of types of segments within the documents. The service may chunk a document into multiple chunks, and operate upon the chunks in parallel by identifying segments within each chunk, classifying the segments into segment types, and processing the segments using special-purpose analysis engines adapted for the analysis of particular segment types to generate results that can be aggregated into an overall output for the entire document that captures the meaning and context of the document text.

Layout-agnostic complex document processing system

Techniques for automated form understanding via layout-agnostic identification of keys and corresponding values are described. An embedding generator creates embeddings of pixels from an image including a representation of a form. The generated embeddings are similar for pixels within a same key-value unit, and far apart for pixels not in a same key-value unit. A weighted bipartite graph is constructed including a first set of nodes corresponding to keys of the form and a second set of nodes corresponding to values of the form. Weights for the edges are determined based on an analysis of distances between ones of the embeddings. The graph is partitioned according to a scheme to identify pairings between the first set of nodes and the second set of nodes that produces a minimum overall edge weight. The pairings indicate keys and values that are associated within the form.

Automated form understanding via layout agnostic identification of keys and corresponding values

Techniques for layout-agnostic clustering-based classification of document keys and values are described. A key-value differentiation unit generates feature vectors corresponding to text elements of a form represented within an electronic image using a machine learning (ML) model. The ML model was trained utilizing a loss function that separates keys from values. The feature vectors are clustered into at least two clusters, and a cluster is determined to include either keys of the form or values of the form via identifying neighbors between feature vectors of the cluster(s) with labeled feature vectors.

Layout-agnostic clustering-based classification of document keys and values

Image data illumination detection

Determining direction of illumination

Described is a multiple-imaging device system and process for calibrating each of the imaging devices to a global color space so that image pixel values representative of an imaged object are the same or similar regardless of the imaging device that produced the image data or the lighting conditions surrounding the object.

Calibration of networked imaging devices to a global color space

A depth detection apparatus is described which has a memory storing raw time-of-flight sensor data received from a time-of-flight sensor. The depth detection apparatus also has a trained machine learning component having been trained using training data pairs. A training data pair comprises at least one simulated raw time-of-flight sensor data value and a corresponding simulated ground truth depth value. The trained machine learning component is configured to compute in a single stage, for an item of the stored raw time-of-flight sensor data, a depth value of a surface depicted by the item, by pushing the item through the trained machine learning component.

Depth from time-of-flight using machine learning

A depth detection apparatus is described which has a memory and a computation logic. The memory stores frames of raw time-of-flight sensor data received from a time-of-flight sensor, the frames having been captured by a time-of-flight camera in the presence of motion such that different ones of the frames were captured using different locations of the camera and/or with different locations of an object in a scene depicted in the frames. The computation logic has functionality to compute a plurality of depth maps from the stream of frames, whereby each frame of raw time-of-flight sensor data contributes to more than one depth map.

Temporal time-of-flight

An embodiment of the invention provides a time of flight three-dimensional TOF-3D camera that determines distance to features in a scene responsive to amounts of light from the scene registered by pixels during different exposure periods and an experimentally determined probabilistic model of how much light the pixels are expected to register during each of the different exposure periods.

Determining distances by probabilistic time of flight imaging

Examples of time-of-flight ('TOF') simulation of multipath light phenomena are described. For example, in addition to recording light intensity for a pixel during rendering, a graphics tool records the lengths (or times) and segment counts for light paths arriving at the pixel. Such multipath information can provide a characterization of the temporal light density of light that arrives at the pixel in response to one or more pulses of light. The graphics tool can use stratification and/or priority sampling to reduce variance in recorded light path samples. Realistic, physically-accurate simulation of multipath light phenomena can, in turn, help calibrate a TOF camera so that it more accurately estimates the depths of real world objects observed using the TOF camera. Various ways to improve the process of inferring imaging conditions such as depth, reflectivity, and ambient light based on images captured using a TOF camera are also described.

Growing community of inventors

Haifa, Israel

Amit Adam