Amazon Technologies, Inc.

A9.com, Inc.

Rahul Bhotika

Avinash Aghoram Ravichandran

Stefano Soatto

Amit Adam

Shai Mazor

Hadar Averbuch Elor

Oleg Rybakov

Rakesh Madhavan Nambiar

Oron Anschel

Andrea Olgiati

Meng Wang

Vijay Mahadevan

Gurumurthy Swaminathan

Joaquin Zepeda Salvatierra

Gal Sabina Star

Lixin Duan

Song Cao

Or Perel

Wendy Tse

Ranju Das

Barath Balasubramanian

Jonathan Wu

Prakash Krishnan

Junxiong Jia

Matias Omar Gregorio Benitez

Hao Wu

Shahar Tsiper

Kunwar Yashraj Singh

Scott Daniel Helmer

Niels Brouwers

Anant Patel

Shaun Ryan James McDowell

Michael Quang Thai Lam

Anshul Kumar Jain

Jian Dong

Anushri Mainthia

Colin Jon Taylor

Pietro Perona

Hao Chen

Bhavesh Anil Doshi

Joseph Patrick Tighe

Mark Scott Waldo

Ismet Zeki Yalniz

Michael Patrick Cutter

Chun-Kai Wang

Paulo Ricardo Dos Santos Mendonca

Anand Dhandhania

Daniya Zamalieva

Umar Farooq

Kaustav Kundu

Patrick Ian Wilson

Hao Yang

Roee Litman

Mohammed El Hamalawi

Tz-Ying Wu

Pashmeen Mistry

Gururaj Kosuru

Guneet Singh Dhillon

Yuting Zhang

Gangadhar Payyavula

Viet Cuong Nguyen

Hao Li

Xinyu Li

Erhan Bas

Omri Ben-Eliezer

Zhizhong Li

Shaun Ryan James Mcdowell

Techniques for class-incremental learning with pretrained machine learning models are described. According to some examples, a pretrained machine learning model that is capable of recognizing objects belonging to a set of object classes is extended to add capability for recognizing additional object classes, without significantly reducing the accuracy of the model for recognizing objects belonging to any of the object classes in the original set of object classes (aka catastrophic forgetting).

Class-incremental learning with pretrained machine learning models

Techniques for feedback-based training may include selecting a scoring machine learning model based at least in part on a test metric, and applying the model on an unlabeled dataset to generate, per dataset item of the unlabeled dataset, a prediction and an importance ranking score for the prediction. Techniques for feedback-based training may further include selecting, based on the importance ranking scores, a result of the application of the model on the unlabeled dataset, providing the result and requesting feedback on the result via a graphical user interface, receiving the feedback via the graphical user interface, adding data from the unlabeled dataset into a training dataset when the feedback indicates a verified result, and retraining the model using the training dataset with the data added from the unlabeled dataset to generate a retrained model.

Feedback-based training for anomaly detection

Techniques for anomaly detection are described. An exemplary method includes receiving one or more requests to train an anomaly detection machine learning model using feedback-based training, the request to indicate one or more of a type of analysis to perform, a model selection indication, and a configuration for a training dataset; training the anomaly detection machine learning model according to the one or more requests using the training data; performing feedback-based training on the trained anomaly detection machine learning model; and using the retrained anomaly detection machine learning model.

Anomaly detection using feedback training

Techniques for creating machine learning models for direct homography regression for image rectification are described. In certain embodiments, a training service trains an algorithm on a source view of a training image and a homography matrix of the training image into a machine learning model that generates a normalized homography matrix for an input of the source view. The normalized homography matrix may then be utilized to generate a target view of an image input into the machine learning model. The target view of the image may be used in a document processing pipeline for document images captured using cameras.

Machine learning models for direct homography regression for image rectification

This disclosure describes automatically selecting and training one or more models for image recognition based upon training and testing (validation) data provided by a user. A service provider network includes a recognition service that may use models to process images and videos to recognize objects in the images and videos, features on the objects in the images and videos, and/or locate objects in the images and videos. The service provider network also includes a model selection and training service that may select one or more modeling techniques based on the objectives of the user and/or the amount of data provided by the user. Based on the selected modeling technique, the model selection and training service selects and trains one or more models for use by the recognition service to process images and videos using the training data. The trained model may be tested and validated using the testing data.

Automated model selection for network-based image recognition service

Methods, systems, and computer-readable media for content moderation using object detection and image classification are disclosed. A content moderation system performs object detection on an input image using one or more object detectors. The object detection finds one or more elements in the input image. The content moderation system performs classification based at least in part on the input image using one or more image classifiers. The classification determines one or more values indicative of one or more content types in the input image. The content moderation system determines one or more scores for one or more content labels corresponding to the one or more content types. At least one of the scores represents a finding of one or more of the content types in the input image. The content moderation system generates output indicating the finding of the one or more of the content types.

Content moderation using object detection and image classification

Various approaches discussed herein enable browsing groups of visually similar items to an item of interest, wherein the item of interest may be identified in a query image, for example. One or more visual attributes associated with the item of interest are identified, and the visually similar items matching at least one of the visual attributes are grouped together, wherein the group is ranked according to the visually similar items' overall visual similarity to the item of interest, for example by using a visual similarity score and/or metric.

Image similarity-based group browsing

Techniques for document rectification via homography recovery using machine learning are described. An image rectification system can intelligently make use of multiple pipelines for rectifying document images based on the detected type of device that generated the images. The image rectification system can provide high-quality rectifications without requiring human cooperation, multiple views of the document in multiple images, and/or without being constrained to only be able to process images from one source context.

Document rectification via homography recovery using machine learning

Techniques for auto-generation of annotated real-world training data are described. An electronic document is analyzed to determine text represented in the document and corresponding locations of the text. A representation of the electronic document is modified to include markers and printed. The printed document is photographed in real-world environments, and the markers within the digital photographs are analyzed to allow for the depiction of the document within the photographs to be rectified. The text and location data are used to annotate the rectified images.

Auto-annotation techniques for text localization

Techniques for grammar-based automated generation of annotated synthetic form training data for machine learning are described. A training data generation engine utilizes a defined grammar to construct a layout for a form, select key-value units to place within the layout, and select attribute variants for the key-value units. The form is rendered and stored at a storage location, where it can be provided along with other similarly-generated forms to be used as training data for a machine learning model.

Grammar-based automated generation of annotated synthetic form training data for machine learning

Techniques for training an embedding using a limited training set are described. In some examples, the embedding is trained by generating a plurality of vectors from a random sample of the limited set of training data classes using a layer of the particular machine learning classification model, randomly selecting samples from the plurality of vectors into a set of samples, computing at least one distance for each sampled class from a center parameter for the class using the set of samples, generating a discrete probability distribution over the classes for a query point based on distances to a center parameter for each of the classes in the embedding space, calculating a loss value for the modified prototypical network, the calculation of the loss value being for a fixed geometry of the embedding space and including a measure of the difference between distributions, and back propagating.

Prototypical network algorithms for few-shot learning

Techniques for layout-agnostic complex document processing are described. A document processing service can analyze documents that do not adhere to defined layout rules in an automated manner to determine the content and meaning of a variety of types of segments within the documents. The service may chunk a document into multiple chunks, and operate upon the chunks in parallel by identifying segments within each chunk, classifying the segments into segment types, and processing the segments using special-purpose analysis engines adapted for the analysis of particular segment types to generate results that can be aggregated into an overall output for the entire document that captures the meaning and context of the document text.

Layout-agnostic complex document processing system

Techniques for automated form understanding via layout-agnostic identification of keys and corresponding values are described. An embedding generator creates embeddings of pixels from an image including a representation of a form. The generated embeddings are similar for pixels within a same key-value unit, and far apart for pixels not in a same key-value unit. A weighted bipartite graph is constructed including a first set of nodes corresponding to keys of the form and a second set of nodes corresponding to values of the form. Weights for the edges are determined based on an analysis of distances between ones of the embeddings. The graph is partitioned according to a scheme to identify pairings between the first set of nodes and the second set of nodes that produces a minimum overall edge weight. The pairings indicate keys and values that are associated within the form.

Automated form understanding via layout agnostic identification of keys and corresponding values

Techniques for keypoint-based multi-label word segmentation and localization are described. A machine learning model identifies bounding regions of text within an image, and then generates multiple channel matrices representing predicted keypoints of the text within the bounding regions. The keypoints can be used to rectify the corresponding graphical content from the image including the text to improve the ability to perform optical character recognition and identify the text. Line and word segmentation and localization can be performed together.

Keypoint-based multi-label word segmentation and localization

Techniques for layout-agnostic clustering-based classification of document keys and values are described. A key-value differentiation unit generates feature vectors corresponding to text elements of a form represented within an electronic image using a machine learning (ML) model. The ML model was trained utilizing a loss function that separates keys from values. The feature vectors are clustered into at least two clusters, and a cluster is determined to include either keys of the form or values of the form via identifying neighbors between feature vectors of the cluster(s) with labeled feature vectors.

Layout-agnostic clustering-based classification of document keys and values

Growing community of inventors

Bellevue, WA, United States of America

Rahul Bhotika