Learning quantization levels

Abstract

A method of reducing data transmission between neural networks in a distributed or federated learning environment, includes the steps of: training a quantization neural network by using a plurality of training vectors each having a dimension k, wherein the quantization neural network is configured to, based on said training, output quantization levels for approximating input vectors having the dimension k; after training the quantization neural network, randomly sampling coordinates of a vector having a dimension d, to provide a first set of k coordinates, wherein d is greater than k; inputting the first set of k coordinates to the quantization neural network to determine first quantization levels for approximating the first set of k coordinates; quantizing the vector having the dimension d based on the determined first quantization levels; and using the quantized vector in the distributed or federated learning environment.