Amazon Technologies, Inc.

Other

University of California

Stefano Soatto

Rahul Bhotika

Amit Adam

Shai Mazor

Oron Anschel

Hadar Averbuch Elor

Wei Xia

Avinash Aghoram Ravichandran

Yuanjun Xiong

Or Perel

Andrea Olgiati

Vijay Mahadevan

Taehee Lee

Shuo Yang

Yifan Xing

Wendy Tse

Gal Sabina Star

Meng Wang

Pietro Perona

Wei Li

Zheng Zhang

Alexander Johannes Smola

Gurumurthy Swaminathan

Paulo Ricardo Dos Santos Mendonca

Shahar Tsiper

Hao Yang

Siqi Deng

Giuseppe Torresin

Tong He

Hao Zhou

Yongxin Wang

Tianjun Xiao

David Paul Wipf

Pratik Anil Chaudhari

Zhizhong Li

Tz-Ying Wu

Pashmeen Mistry

Guneet Singh Dhillon

Rasool Fakoor

Omri Ben-Eliezer

Yuting Zhang

Gangadhar Payyavula

Viet Cuong Nguyen

Yantao Shen

Techniques for class-incremental learning with pretrained machine learning models are described. According to some examples, a pretrained machine learning model that is capable of recognizing objects belonging to a set of object classes is extended to add capability for recognizing additional object classes, without significantly reducing the accuracy of the model for recognizing objects belonging to any of the object classes in the original set of object classes (aka catastrophic forgetting).

Class-incremental learning with pretrained machine learning models

Techniques for Meta-Q-Learning (MQL) are described. A method of MQL may include receiving a request from an agent to perform adaptation based at least on task data associated with a new task collected by the agent, identifying a subset of meta-training data corresponding to the task data in a replay buffer, and adapting a policy using the subset of meta-training data and the task data to generate an adapted policy, wherein the adapted policy is used identify a next action for the agent to perform.

Meta-Q learning

Machine learning models may be generated that are compatible with another machine learning model and satisfy a resource constraint. Techniques that ensure weight compatibility and architectural compatibility between a machine learning model being created to be compatible with another machine learning model are applied. The resource constraint is enforced so that the generated machine learning model also fits within the resource constraint.

Compatible machine learning model generation within resource constraints

Techniques for performing visual clustering with a hierarchical graph neural network framework including a joint linkage prediction and density estimation graph model are described. Embodiments herein recurrently run the joint linkage prediction and density estimation graph model to generate intermediate clusters in multiple iterations (e.g., until convergence) to obtain a final clustering result. In certain embodiments, for each iteration, the input graph contains nodes that are merged from nodes assigned to intermediate clusters from the previous iteration. By using a small and fixed bandwidth k in each iteration, embodiments herein alleviate the sensitivity to the k selection for different clustering applications. Certain embodiments herein remove the tuning of a different k (e.g., k-bandwidth) for k-nearest neighbor graph construction over different clustering applications.

Hierarchical graph neural networks for visual clustering

This disclosure describes automatically selecting and training one or more models for image recognition based upon training and testing (validation) data provided by a user. A service provider network includes a recognition service that may use models to process images and videos to recognize objects in the images and videos, features on the objects in the images and videos, and/or locate objects in the images and videos. The service provider network also includes a model selection and training service that may select one or more modeling techniques based on the objectives of the user and/or the amount of data provided by the user. Based on the selected modeling technique, the model selection and training service selects and trains one or more models for use by the recognition service to process images and videos using the training data. The trained model may be tested and validated using the testing data.

Automated model selection for network-based image recognition service

Techniques for auto-generation of annotated real-world training data are described. An electronic document is analyzed to determine text represented in the document and corresponding locations of the text. A representation of the electronic document is modified to include markers and printed. The printed document is photographed in real-world environments, and the markers within the digital photographs are analyzed to allow for the depiction of the document within the photographs to be rectified. The text and location data are used to annotate the rectified images.

Auto-annotation techniques for text localization

Techniques for building a backward compatible and backfill-free image search system are described. According to some embodiments, a backwards compatible training system trains a new embedding model to be backward compatible with the face embeddings (e.g., floating-point vectors) generated by a previous embedding model. In one embodiment, backwards compatible training uses a classifier of the previous embedding model as a form of constraint in the training of the new embedding model.

Backward compatible and backfill-free image search system

Techniques for grammar-based automated generation of annotated synthetic form training data for machine learning are described. A training data generation engine utilizes a defined grammar to construct a layout for a form, select key-value units to place within the layout, and select attribute variants for the key-value units. The form is rendered and stored at a storage location, where it can be provided along with other similarly-generated forms to be used as training data for a machine learning model.

Grammar-based automated generation of annotated synthetic form training data for machine learning

Techniques for training an embedding using a limited training set are described. In some examples, the embedding is trained by generating a plurality of vectors from a random sample of the limited set of training data classes using a layer of the particular machine learning classification model, randomly selecting samples from the plurality of vectors into a set of samples, computing at least one distance for each sampled class from a center parameter for the class using the set of samples, generating a discrete probability distribution over the classes for a query point based on distances to a center parameter for each of the classes in the embedding space, calculating a loss value for the modified prototypical network, the calculation of the loss value being for a fixed geometry of the embedding space and including a measure of the difference between distributions, and back propagating.

Prototypical network algorithms for few-shot learning

Techniques for automated form understanding via layout-agnostic identification of keys and corresponding values are described. An embedding generator creates embeddings of pixels from an image including a representation of a form. The generated embeddings are similar for pixels within a same key-value unit, and far apart for pixels not in a same key-value unit. A weighted bipartite graph is constructed including a first set of nodes corresponding to keys of the form and a second set of nodes corresponding to values of the form. Weights for the edges are determined based on an analysis of distances between ones of the embeddings. The graph is partitioned according to a scheme to identify pairings between the first set of nodes and the second set of nodes that produces a minimum overall edge weight. The pairings indicate keys and values that are associated within the form.

Automated form understanding via layout agnostic identification of keys and corresponding values

Techniques for layout-agnostic clustering-based classification of document keys and values are described. A key-value differentiation unit generates feature vectors corresponding to text elements of a form represented within an electronic image using a machine learning (ML) model. The ML model was trained utilizing a loss function that separates keys from values. The feature vectors are clustered into at least two clusters, and a cluster is determined to include either keys of the form or values of the form via identifying neighbors between feature vectors of the cluster(s) with labeled feature vectors.

Layout-agnostic clustering-based classification of document keys and values

A structured document analyzer that associates keys and values in structured documents based on key, value, and key-value container bounding boxes. A trained machine learning model analyzes images of structured documents to determine bounding boxes for keys, values, and key-value containers in the images with confidence scores for the classifications. For each image, duplicate bounding boxes are removed, and then a set of key-value containers are selected and sorted based on the confidence scores. For each key-value container, a best key and value are determined for the container based on overlap of the key and value bounding boxes with the container bounding box and the confidence scores. Optical character recognition may be performed on the image to determine text for the keys and values.

Structured document analyzer

Techniques for multiple object tracking in video are described in which the outputs of neural networks are combined within a Bayesian framework. A motion model is applied to a probability distribution representing the estimated current state of a target object being tracked to predict the state of the target object in the next frame. A state of an object can include one or more features, such as the location of the object in the frame, a velocity and/or acceleration of the object across frames, a classification of the object, etc. The prediction of the state of the target object in the next frame is adjusted by a score based on the combined outputs of neural networks that process the next frame.

Multiple object tracking in video by combining neural networks within a bayesian framework

A disposable volume spirometry apparatus and non-contact measurement system is described. The spirometry system includes an expandable disposable volume spirometry apparatus, a remote non-contact sensor, memory, and a processor. The remote non-contact sensor captures images associated with the expandable disposable volume spirometry apparatus. The memory stores the captured images. The processor is operatively coupled to the memory and determines a volume for the expandable disposable volume spirometry apparatus by analyzing the captured images.

Spirometry system and disposable volume spirometry apparatus

We describe an end-to-end visual recognition system, where 'end-to-end' refers to the ability of the system of performing all aspects of the system, from the construction of 'maps' of scenes, or &#x201c;models&#x201d; of objects from training data, to the determination of the class, identity, location and other inferred parameters from test data. Our visual recognition system is capable of operating on a mobile hand-held device, such as a mobile phone, tablet or other portable device equipped with sensing and computing power. Our system employs a video based feature descriptor, and we characterize its invariance and discriminative properties. Feature selection and tracking are performed in real-time, and used to train a template-based classifier during a capture phase prompted by the user. During normal operation, the system scores objects in the field of view based on their ranking.

End-to end visual recognition system and methods

Growing community of inventors

Pasadena, CA, United States of America

Stefano Soatto