Instruction-based performance prediction for deep learning accelerator

Abstract

Aspects of the present disclosure provide a method for training a predictor that predicts performance of a plurality of machine learning (ML) models on platforms. For example, the method can include converting each of the ML models into a plurality of instructions or the instructions and a plurality of intermediate representations (IRs). The method can also include simulating execution of the instructions corresponding to each of the ML models on a platform and generating instruction performance reports. Each of the instruction performance reports can be associated with performance of the instructions corresponding to one of the ML models that are executed on the platform. The method can also include training the predictor with the instructions or the IRs as learning features and the instruction performance reports as learning labels, compiling the predictor into a library file, and storing the library file in a storage device.