Huawei Technologies Co., Limited

Serdar Sozubek

Barnaby Dalton

Vanessa Courville

Manuel Saldana

Graham Taylor

The present disclosure is drawn to the reduction of parameters in fully connected layers of neural networks. For a layer whose output is defined by y=Wx, where y&#x2208;Ris the output vector, x&#x2208;Ris the input vector, and W&#x2208;Ris a matrix of connection parameters, matrices Uand Vare defined and submatrices Ware computed as the product of Uand V, so that W=VU, and W is obtained by appending submatrices W.

Reduction of parameters in fully connected layers of neural networks by low rank factorizations

The present disclosure is drawn to the reduction of parameters in fully connected layers of neural networks. For a layer whose output is defined by y=Wx, where y is the output vector, x is the input vector, and W is a matrix of connection parameters, vectors uand vare defined and submatrices Ware computed as the outer product of uand v, so that W=v&#x2297;u, and W is obtained by appending submatrices W.

Growing community of inventors

North York, Canada

Serdar Sozubek