Amazon Technologies, Inc.

Vikram Madan

Rahul Sharma

Hagay Lupesko

Sandeep Krishnamurthy

Roshani Nagmote

Yao Wang

Jonathan Esterhazy

Naveen Mysore Nagendra Swamy

Dominic Rajeev Divakaruni

James Robert Blair

Charles H Bell

Arvind Jayasundar

Jiajie Chen

Ruofei Yu

Hareesh Lakshmi Narayanan

Techniques for active learning-based data labeling are described. An active learning-based data labeling service enables a user to build and manage large, high accuracy datasets for use in various machine learning systems. Machine learning may be used to automate annotation and management of the datasets, increasing efficiency of labeling tasks and reducing the time required to perform labeling. Embodiments utilize active learning techniques to reduce the amount of a dataset that requires manual labeling. As subsets of the dataset are labeled, this label data is used to train a model which can then identify additional objects in the dataset without manual intervention. The process may continue iteratively until the model converges. This enables a dataset to be labeled without requiring each item in the data set to be individually and manually labeled by human labelers.

Custom labeling workflows in an active learning-based data labeling service

Techniques for model adaptation are described. For example, a method of receiving a call to provide either a model variant or a model variant profile of a deep learning model, the call including desired performance of the deep learning model, a deep learning model identifier, and current edge device characteristics; comparing the received current edge device characteristics to available model variants and profiles based on the desired performance of the deep learning model to generate or select a model variant or profile, the available model variants and profiles determined by the model identifier; and sending the generated or selected model variant or profile to the edge device to use in inference is detailed.

Model adaptation

Techniques for generating and utilizing machine learning based adaptive instructions for annotation are described. An annotation service can use models to identify edge case data elements predicted to elicit differing annotations from annotators, 'bad' data elements predicted to be difficult to annotate, and/or 'good' data elements predicted to elicit matching or otherwise high-quality annotations from annotators. These sets of data elements can be automatically incorporated into annotation job instructions provided to annotators, resulting in improved overall annotation results via having efficiently and effectively &#x201c;trained&#x201d; the annotators how to perform the annotation task.

Machine learning based adaptive instructions for annotation

Techniques for benchmarking a machine learning model/algorithm are described. For example, in some instances a method includes generating an execution plan for benchmarking of at least one task corresponding to a machine learning model based on an identified machine learning model, identified training data, and at least one objective for the benchmarking job; receiving execution statistics about the execution of the task as a part of the benchmarking job according to the execution plan; and updating the execution plan based at least in part on the received execution statistics of the task.

Benchmarking machine learning models via performance feedback

Growing community of inventors

San Francisco, CA, United States of America

Vikram Madan