Amazon Technologies, Inc.

Poorna Chand Srinivas Perumalla

Stefano Stefani

Andrea Olgiati

Pracheer Gupta

Nagajyothi Nookula

Sudipta Sengupta

Matthew James Wood

Madan Mohan Rao Jampani

Swaminathan Sivasubramanian

Nafea Bshara

Leo Parker Dirac

Eduardo Manuel Calleja

Dominic Rajeev Divakaruni

Bratin Saha

Vinay Hanumaiah

Aashish Jindia

Jia Bi Zhang

Srikanth Kandalam Srinivasa

Wei Xiao

Matthew Shawn Wilson

Rahul Sharma

David Robert Thomas

Pejus Manoj Das

Long Gao

Haifeng He

Safeer Mohiuddin

Jalaja C Kurubarahalli

Pragya Agarwal

Romi Boimer

Jiacheng Guo

Rajankumar Singh

Vladimir Mitrovic

Fei Chen

Wanqiang Chen

Song Ge

Danjuan Ye

Maden Mohan Rao Jampani

Zdravko Pantic

Mark Klunder

Umang Wadhwa

Daniel Evans

Karan Kothari

Eftiquar Shaikh

Jun Wu

Cory Pruce

Samuel Oshin

Shirley Xue Yi Leung

Yongcong Luo

Techniques for machine learning (ML) model adaptation via segment replacement and student-teacher training are described. A model optimizer determines that a ML model sought to be deployed to an edge device includes a source segment that is not supported by the edge device. The model optimizer identifies a replacement segment that is equivalent to the source segment, constructs an adapted ML model by swapping in the replacement segment for the source segment, and trains the adapted ML model, optionally from the source ML model using a student-teacher training procedure. The trained adapted ML model is then deployed to the edge device, where it can be successfully run with minimal (if any) degradation of performance compared to the original source ML model.

Machine learning model adaptation via segment replacement and student-teacher training

Techniques for input adaptation from disparate data sources for heterogeneous machine learning model execution are described. A preprocessing adapter can perform preprocessing of data obtained from edge devices to suit the input data characteristic requirements of one or more machine learning (ML) models. The preprocessing adapter can determine the input data characteristic requirements in a variety of ways, such as via analysis of the input layer of a ML model or through data variation testing and associated feedback resulting from output data generated by the ML model.

Intelligent input adaptation from disparate data sources for heterogeneous machine learning model execution

Implementations detailed herein include description of a computer-implemented method. In an implementation, the method at least includes attaching a first set of one or more accelerator slots of an accelerator appliance to an application instance of a multi-tenant provider network according to an application instance configuration, the application instance configuration to define per accelerator slot capabilities to be used by an application of the application instance, wherein the multi-tenant provider network comprises a plurality of computing devices configured to implement a plurality of virtual compute instances, and wherein the first set of one or more accelerator slots is implemented using physical accelerator resources accessible to the application instance; while performing inference using the loaded machine learning model of the application using the first set of one or more accelerator slots on the attached accelerator appliance, managing resources of the accelerator appliance using an accelerator appliance manager of the accelerator appliance.

Fault-tolerant accelerator based inference service

Techniques for machine learning (ML) model training and deployment using model building blocks via graphical user interfaces (GUIs) are described. Users can use a GUI provided by an electronic device to select and configure ML aspects for one or more ML models to be trained using identified training data. The electronic device can send a request to cause a model construction service to train one or more ML models based on the user configuration, return results of the training to the user within the GUI, and deploy one or more of the ML models.

Machine learning model creation via user-configured model building blocks

Techniques for generating and executing an execution plan for a machine learning (ML) model using one of an edge device and a non-edge device are described. In some examples, a request for the generation of the execution plan includes at least one objective for the execution of the ML model and the execution plan is generated based at least in part on comparative execution information and network latency information.

Automated distribution of models for execution on a non-edge device and an edge device

A model selection and deployment service at a provider network receives an indication of sensor availability from a remote client device (e.g., what type of sensors are currently available to provide sensor data to the client device). The model selection and deployment service then selects, based on the sensor availability (and/or based on one or more other factors/criteria), a data processing model from a group of data processing models that are available for deployment to the client device. The model selection and deployment service then transmits the selected data processing model to the remote client device (e.g., for installation on the hub device). This may allow a client device to use the best data processing model for a sensor configuration and to dynamically adjust to any changes in the sensor configuration.

Selecting and deploying models based on sensor availability

Implementations detailed herein include description of a computer-implemented method. In an implementation, the method at least includes receiving an application instance configuration, an application of the application instance to utilize a portion of an attached accelerator during execution of a machine learning model and the application instance configuration including: an indication of the central processing unit (CPU) capability to be used, an arithmetic precision of the machine learning model to be used, an indication of the accelerator capability to be used, a storage location of the application, and an indication of an amount of random access memory to use.

Attached accelerator based inference service

Techniques for utilizing adaptable filters for edge-based deep learning models are described. Filters may be utilized by an edge electronic device to filter elements of an input data stream so that only a subset of the elements are used as inputs to a machine learning model run by the electronic device, enabling successful operation despite the input data stream potentially being generated at a higher rate than a rate in which the ML model can be executed. The filter can be a differential-type filter that generates difference representations between consecutive elements of the data stream to determine which elements are to be passed on for the ML model, a 'smart' filter such as a neural network trained using outputs from the ML model allowing the filter to 'learn' which elements are the most likely to be of value to be passed on, or a combination of both.

Adaptable filtering for edge-based deep learning models

Implementations detailed herein include description of a computer-implemented method. In an implementation, the method at least includes receiving an application instance configuration, an application of the application instance to utilize a portion of an attached accelerator during execution of a machine learning model and the application instance configuration including an arithmetic precision of the machine learning model to be used in determining the portion of the accelerator to provision; provisioning the application instance and the portion of the accelerator attached to the application instance, wherein the application instance is implemented using a physical compute instance in a first location, wherein the portion of the accelerator is implemented using a physical accelerator in the second location; loading the machine learning model onto the portion of the accelerator; and performing inference using the loaded machine learning model of the application using the portion of the accelerator on the attached accelerator.

Attached accelerator selection and placement

Techniques for partitioning data flow operations between execution on a compute instance and an attached accelerator instance are described. A set of operations supported by the accelerator is obtained. A set of operations associated with the data flow is obtained. An operation in the set of operations associated with the data flow is identified based on the set of operations supported by the accelerator. The accelerator executes the first operation.

Framework integration for instance-attachable accelerator

Implementations detailed herein include description of a computer-implemented method. In an implementation, the method at least includes provisioning an application instance and portions of at least one accelerator attached to the application instance to execute a machine learning model of an application of the application instance; loading the machine learning model onto the portions of the at least one accelerator; receiving scoring data in the application; and utilizing each of the portions of the attached at least one accelerator to perform inference on the scoring data in parallel and only using one response from the portions of the accelerator.

Attached accelerator scaling

A processing device receives a dataset comprising a plurality of data points, wherein each data point of the plurality of data points comprises a representative vector for the data point and an associated classification for the data point. The processing device determines, for the dataset, a score representative of a degree of clustering of the plurality of data points. The processing device determines a suitability of the dataset for use in machine learning based on the score.

Dataset suitability check for machine learning

Techniques are described for a nearest neighbor search service that enables users to perform nearest neighbor searches. The nearest neighbor search service includes an interface that enables users to create collections of searchable vectors, add and update vectors to a collection, delete vectors from a collection, and perform searches for nearest neighbors to a given vector. The nearest neighbor search service enables users to add, update, and delete vectors of a collection in real-time while also enabling users to perform searches at the same time.

Nearest neighbor search as a service

A data set may be partitioned according to relative differences indicated by a cover tree. A cover tree may be generated for a data set. Items in the data set may be stored at the same or different nodes according to the relative difference between the items indicated in the cover tree. Portions of the cover tree may be assigned to different nodes storing the data set. Access requests for the data set may be performed by sending the access requests to nodes identified according to the assigned portions of the cover tree.

Partitioning data according to relative differences indicated by a cover tree

Incremental updates for nearest neighbor search

Growing community of inventors

Seattle, WA, United States of America

Poorna Chand Srinivas Perumalla