Amazon Technologies, Inc.

Zhi Chen

Yida Wang

Yizhi Liu

Randy Renfu Huang

Richard John Heaton

Jindrich Zejda

Drazen Borkovic

Jeffrey T Huynh

Hongbin Zheng

Mu Li

Animesh Jain

Haichen Li

Yao Wang

Leyuan Wang

Xingjian Shi

Haichen Shen

Tuning result records indicating tuned schedules for machine learning tuning tasks are stored at a data store accessed from a tuning service. A given schedule indicates at least an order in which sub-operations of a tuning task are to be executed. In response to determining that the data store does not include a result record whose tuning task meets a similarity criterion to a tuning task determined from a tuning request, performance predictions for a set of candidate schedules are obtained using a performance prediction model, without running code which executes the candidate schedules. In a response to the tuning request, executable code corresponding to a preferred schedule identified from the candidate schedules using the performance predictions is included.

Tuned executable code service for machine learning models

Methods and apparatuses for hierarchical partitioning of operators of a neural network for execution on an acceleration engine are provided. Neural networks are built in machine learning frameworks using neural network operators. The neural network operators are compiled into executable code for the acceleration engine. Development of new framework-level operators can exceed the capability to map the newly developed framework-level operators onto the acceleration engine. To enable neural networks to be executed on an acceleration engine, hierarchical partitioning can be used to partition the operators of the neural network. The hierarchical partitioning can identify operators that are supported by a compiler for execution on the acceleration engine, operators to be compiled for execution on a host processor, and operators to be executed on the machine learning framework.

Hierarchical partitioning of operators

Techniques for optimizing and deploying convolutional neural network (CNN) machine learning models for inference using integrated graphics processing units are described. A model compilation system optimizes CNN models using optimized vision-specific operators as well as both graph-level tuning and tensor-level tuning to explore the optimization space for achieving heightened performance. The model compilation system may also implement a heuristic-based two-stage technique for falling back certain operators of CNN models to use CPUs when needed or otherwise beneficial.

Unified optimization for convolutional neural network model inference on integrated graphics processing units

Growing community of inventors

Santa Clara, CA, United States of America

Zhi Chen