Method and apparatus with neural network parameter quantization

Abstract

A processor-implemented neural network method includes: determining a respective probability density function (PDF) of normalizing a statistical distribution of parameter values, for each channel of each of a plurality of feature maps of a pre-trained neural network; determining, for each channel, a corresponding first quantization range for performing quantization of corresponding parameter values, based on a quantization error and a quantization noise of the respective determined PDF; determining, for each channel, a corresponding second quantization range, based on a signal-to-quantization noise ratio (SQNR) of the respective determined PDF; correcting, for each channel, the corresponding first quantization range based on the corresponding second quantization range; and generating a quantized neural network, based on the corrected first quantization range corresponding for each channel.