Neural network model conversion pipeline

Abstract

Techniques are discussed herein for converting neural network models and validating the conversions to maintain accuracy and avoid performance regressions. A model conversion pipeline may execute inference on original and converted neural network models to identify potential accuracy regressions. A layer-by-layer conversion and analysis can be used to determine the errors generated by specific layers or operations within the model. In some examples, conversion and validation processes may be initiated on pre-trained neural networks. Latency data also may be collected for individual layers and compared between the original and converted neural network models. An automated pipeline incorporating various conversion and validation techniques may maintain sets of conversion options for different models, to perform automatic conversion, execution, validation, and deployment of updated neural network models to the cloud or other target systems.