Amazon Technologies, Inc.

Altera Corporation

Andrea Olgiati

Poorna Chand Srinivas Perumalla

Stefano Stefani

Steven L Teig

Swaminathan Sivasubramanian

Nafea Bshara

Sudipta Sengupta

Leo Parker Dirac

Matthew Pond Baker

Matthew James Wood

Lakshmi Naarayanan Ramakrishnan

Dominic Rajeev Divakaruni

Bratin Saha

Ron Diamant

Saurabh Gupta

Rahul Bhotika

Ramyanshu Datta

Amit Adam

Shai Mazor

Vasant Manohar

Pracheer Gupta

Wei You

Stephen Gould

Nitin Singhal

Kohen Berith Chia

Kunal Mehrotra

Arvind Sowmyan

Ishaaq Chandy

Wendy Tse

Nathan Lorenzo Binkert

Stefano Soatto

Madan Mohan Rao Jampani

Vladimir Zhukov

Jia Bi Zhang

Jonathan Hedley

Vivek Bhadauria

Arun Babu Nagarajan

Oron Anschel

Laurence Louis Eric Rouesnel

Hadar Averbuch Elor

Davide Modolo

Gowtham Jeyabalan

Vikas Kumar

Gal Sabina Star

Rahul Raghavendra Huilgol

Or Perel

Anurag Windlass Gupta

Randy Renfu Huang

Meng Wang

Rahul Sharma

Gaurav D Ghare

Richard John Heaton

Maximiliano Maccanti

Hao Chen

Roman Goldenberg

Michail Petropoulos

John Benjamin Tobler

Bhavesh Anil Doshi

David Robert Thomas

Joseph Patrick Tighe

Stavros Harizopoulos

Zohar Karnin

Vikram Singh Bisht

Ranju Das

Gowda Dayananda Anjaneyapura Range

Rakesh Madhavan Nambiar

Anand Dhandhania

Shubham Chandra Gupta

Umar Farooq

Jalaja C Kurubarahalli

Carlos Garcia-Alvarado

Urvashi Chowdhary

Pragya Agarwal

Patrick Ian Wilson

Praveen Kumar Udayakumar

Zhuzeng Lu

Terence Kelly

Enrica Maria Filippi

Rajankumar Singh

Kuang Han

Renjun Zheng

Maden Mohan Rao Jampani

Samuel Oshin

Yuri Natanzon

Cory Pruce

Jun Wu

Eftiquar Shaikh

Daniel Evans

Umang Wadhwa

Mark Klunder

Zdravko Pantic

Denis Davydenko

Jeffrey John Geevarghese

Camille Claire Le Men

Amol Ashok Lele

Zhihan Li

Rahul Iyer

Omri Ben-Eliezer

Gururaj Kosuru

Praveenkumar Udayakumar

A determination is made as to whether a workload associated with a collection of data items satisfies a first condition. If the first condition is satisfied, a determination is made as to whether a metric of dirty records to which delete operations have been directed within a portion of a data item satisfies a second condition. If the second condition is also satisfied, a cleanup operation may be initiated so that data blocks storing the portion of the data item do not include the deleted record.

Background incremental deletion cleanup techniques at storage services

Techniques for adaptive machine learning training via in-flight feature modification are described. A training monitor captures training data during the training of a machine learning model, and a metric generator creates metrics such as feature importance metrics based on the data. A rule evaluation engine determines whether any modification conditions are met for any of the features based on the metrics, and based on such a determination can cause the in-flight training job to be modified.

Adaptive machine learning training via in-flight feature modification

A post-task-completion retention period for which a computing resource is to be retained, without de-activating the resource, on behalf of a set of requesters of machine learning tasks is determined at a machine learning service. A first task, identified at the service prior to expiration of the retention period at a first computing resource at which a second task has completed, is initiated at the first computing resource. In response to obtaining an indication of a third task and determining that a threshold criterion associated with the retention period satisfies a criterion, the third task is initiated at an additional computing resource. The additional computing resource is de-activated after the third task completes, without waiting for the retention period to expire.

Client-configurable retention periods for machine learning service-managed resources

Parameters of a pool of computing resources to be utilized for machine learning tasks from a set of entities are stored, including a category of the computing resources, and a post-task-completion retention period during which, after completion of a task, at least a portion of data stored at the resource is not to be deleted. A compute instance of the pool is assigned to a task requested from the set of entities after determining that one or more configuration settings of the instance satisfy a preference indicated in the request for the task, and that the retention period of the instance relative to a completion of an earlier task on the instance has not expired. A result of the task is stored.

Resource management techniques to reduce startup overhead for machine learning tasks

Methods, systems, and computer-readable media for debugging and profiling of machine learning model training are disclosed. A machine learning analysis system receives data associated with training of a machine learning model. The data was collected by a machine learning training cluster. The machine learning analysis system performs analysis of the data associated with the training of the machine learning model. The machine learning analysis system detects one or more conditions associated with the training of the machine learning model based at least in part on the analysis. The machine learning analysis system generates one or more alarms describing the one or more conditions associated with the training of the machine learning model.

Debugging and profiling of machine learning model training

Implementations detailed herein include description of a computer-implemented method. In an implementation, the method at least includes attaching a first set of one or more accelerator slots of an accelerator appliance to an application instance of a multi-tenant provider network according to an application instance configuration, the application instance configuration to define per accelerator slot capabilities to be used by an application of the application instance, wherein the multi-tenant provider network comprises a plurality of computing devices configured to implement a plurality of virtual compute instances, and wherein the first set of one or more accelerator slots is implemented using physical accelerator resources accessible to the application instance; while performing inference using the loaded machine learning model of the application using the first set of one or more accelerator slots on the attached accelerator appliance, managing resources of the accelerator appliance using an accelerator appliance manager of the accelerator appliance.

Fault-tolerant accelerator based inference service

Systems and methods for performing improper input data detection are described. In one example, a system comprises: hardware circuits configured to receive input data and to perform computations of a neural network based on the input data to generate computation outputs; and an improper input detection circuit configured to: determine a relationship between the computation outputs of the hardware circuits and reference outputs; determine that the input data are improper based on the relationship; and perform an action based on determining that the input data are improper.

Improper neural network input detection and handling

Implementations detailed herein include description of a computer-implemented method. In an implementation, the method at least includes receiving an application instance configuration, an application of the application instance to utilize a portion of an attached accelerator during execution of a machine learning model and the application instance configuration including: an indication of the central processing unit (CPU) capability to be used, an arithmetic precision of the machine learning model to be used, an indication of the accelerator capability to be used, a storage location of the application, and an indication of an amount of random access memory to use.

Attached accelerator based inference service

Implementations detailed herein include description of a computer-implemented method. In an implementation, the method at least includes receiving an application instance configuration, an application of the application instance to utilize a portion of an attached accelerator during execution of a machine learning model and the application instance configuration including an arithmetic precision of the machine learning model to be used in determining the portion of the accelerator to provision; provisioning the application instance and the portion of the accelerator attached to the application instance, wherein the application instance is implemented using a physical compute instance in a first location, wherein the portion of the accelerator is implemented using a physical accelerator in the second location; loading the machine learning model onto the portion of the accelerator; and performing inference using the loaded machine learning model of the application using the portion of the accelerator on the attached accelerator.

Attached accelerator selection and placement

Techniques for partitioning data flow operations between execution on a compute instance and an attached accelerator instance are described. A set of operations supported by the accelerator is obtained. A set of operations associated with the data flow is obtained. An operation in the set of operations associated with the data flow is identified based on the set of operations supported by the accelerator. The accelerator executes the first operation.

Framework integration for instance-attachable accelerator

Methods, systems, and computer-readable media for GPU code injection to summarize machine learning training data are disclosed. Training of a machine learning model is initiated using a graphics processing unit (GPU) associated with a machine learning training cluster. The training of the machine learning model generates tensor data in a memory of the GPU. The GPU determines a summary of the tensor data according to a reduction operator. The summary is smaller in size than the tensor data and is output by the GPU. A machine learning analysis system performs an analysis of the training of the machine learning model based at least in part on the summary of the tensor data. The machine learning analysis system detects one or more conditions associated with the training of the machine learning model based at least in part on the analysis.

GPU code injection to summarize machine learning training data

Methods, systems, and computer-readable media for automated problem detection for machine learning models are disclosed. A machine learning analysis system receives data associated with use of a machine learning model. The data was collected by a machine learning inference system and comprises input to the model or a plurality of inferences representing output of the machine learning model. The machine learning analysis system performs analysis of the data associated with the use of the machine learning model. The machine learning analysis system detects one or more problems associated with the use of the machine learning model based at least in part on the analysis. The machine learning analysis system initiates one or more remedial actions associated with the one or more problems associated with the use of the machine learning model.

Automated problem detection for machine learning models

Implementations detailed herein include description of a computer-implemented method. In an implementation, the method at least includes provisioning an application instance and portions of at least one accelerator attached to the application instance to execute a machine learning model of an application of the application instance; loading the machine learning model onto the portions of the at least one accelerator; receiving scoring data in the application; and utilizing each of the portions of the attached at least one accelerator to perform inference on the scoring data in parallel and only using one response from the portions of the accelerator.

Attached accelerator scaling

A processing device receives a dataset comprising a plurality of data points, wherein each data point of the plurality of data points comprises a representative vector for the data point and an associated classification for the data point. The processing device determines, for the dataset, a score representative of a degree of clustering of the plurality of data points. The processing device determines a suitability of the dataset for use in machine learning based on the score.

Dataset suitability check for machine learning

Growing community of inventors

Gilroy, CA, United States of America

Andrea Olgiati