Samsung Electronics Co., Ltd.

Seoul National University

Junhaeng Lee

Seungwon Lee

Sehwan Lee

Shinhaeng Kang

Joonho Song

Wonjo Lee

Sangwon Ha

Hyunsun Park

Sungjoo Yoo

Eunhyeok Park

Yeongjae Choi

SeungWon Lee

A neural network method and apparatus are provided. A processor implemented neural network includes calculating respective individual gradient values for updating a weight of a neural network, calculating a residual gradient value based on an accumulated gradient value obtained by accumulating the individual gradient values and a bit digit representing the weight, tuning the respective individual gradient values to correspond to a bit digit of the residual gradient value, summing the tuned respective individual gradient values, the residual gradient value, and the weight, and updating the weight and the residual gradient value based on a result of the summing to train the neural network.

Neural network method and apparatus

Provided are a neural network method and an apparatus, the method including obtaining a set of floating point data processed in a layer included in a neural network, determining a weighted entropy based on data values included in the set of floating point data, adjusting quantization levels assigned to the data values based on the weighted entropy, and quantizing the data values included in the set of floating point data in accordance with the adjusted quantization levels.

According to a method and apparatus for neural network quantization, a quantized neural network is generated by performing learning of a neural network, obtaining weight differences between an initial weight and an updated weight determined by the learning of each cycle for each of layers in the first neural network, analyzing a statistic of the weight differences for each of the layers, determining one or more layers, from among the layers, to be quantized with a lower-bit precision based on the analyzed statistic, and generating a second neural network by quantizing the determined one or more layers with the lower-bit precision.

Method and apparatus for neural network quantization

A method of generating a fixed-point quantized neural network includes analyzing a statistical distribution for each channel of floating-point parameter values of feature maps and a kernel for each channel from data of a pre-trained floating-point neural network, determining a fixed-point expression of each of the parameters for each channel statistically covering a distribution range of the floating-point parameter values based on the statistical distribution for each channel, determining fractional lengths of a bias and a weight for each channel among the parameters of the fixed-point expression for each channel based on a result of performing a convolution operation, and generating a fixed-point quantized neural network in which the bias and the weight for each channel have the determined fractional lengths.

Method and apparatus for generating fixed-point quantized neural network

A method of quantizing parameters of a neural network includes calculating, for each of the parameters, a bit shift value indicating a degree outside a bit range of a fixed-point format for quantizing the parameters, updating the fixed-point format based on the calculated bit shift values of the parameters, and quantizing parameters updated in a learning or inference process according to the updated fixed-point format.

Method and apparatus for quantizing parameters of neural network

A neural network method and apparatus is provided. A processor-implemented neural network method includes determining, based on a determined number of classes of input data, a precision for a neural network layer outputting an operation result, and processing parameters of the layer according to the determined precision.

A neural network method of parameter quantization obtains channel profile information for first parameter values of a floating-point type in each channel included in each of feature maps based on an input in a first dataset to a floating-point parameters pre-trained neural network, and determines a probability density function (PDF) type, for each channel, appropriate for the channel profile information based on a classification network receiving the channel profile information as a dataset. The neural network method of parameter quantization determines a fixed-point representation, based on the determined PDF type, for each channel, statistically covering a distribution range of the first parameter values, and generates a fixed-point quantized neural network based on the fixed-point representation determined for each channel.

Neural network method and apparatus with parameter quantization

A processor-implemented neural network method includes calculating individual update values for a weight assigned to a connection relationship between nodes included in a neural network; generating an accumulated update value by accumulating the individual update values in an accumulation buffer; and training the neural network by updating the weight using the accumulated update value in response to the accumulated update value being equal to or greater than a threshold value.

Growing community of inventors

Hwaseong-si, South Korea

Junhaeng Lee