Hardware environment-based data quantization method and apparatus, and readable storage medium

Abstract

A hardware environment-based data quantization method includes: parsing a model file under a current deep learning framework to obtain intermediate computational graph data and weight data that are independent of a hardware environment; performing calculation on image data in an input data set through a process indicated by an intermediate computational graph to obtain feature map data; separately performing uniform quantization on the weight data and the feature map data of each layer according to a preset linear quantization method, and calculating a weight quantization factor and a feature map quantization factor (S); combining the weight quantization factor and the feature map quantization factor to obtain a quantization parameter that makes hardware use shift instead of division; and finally, writing the quantization parameter and the quantized weight data to a bin file according to a hardware requirement so as to generate quantized file data (S).