Huawei Technologies Co., Limited

Vanessa Courville

Barnaby Dalton

Manuel Saldana

Serdar Sozubek

Graham Taylor

Sepideh Kharaghani

A circuit for a multistage sequential data process includes a plurality of memory units. Each memory unit is associated with a stage of the sequential data process which, for each data set inputted to the stage, the stage provides an intermediate data set for storage in the associated memory unit for use in at least one subsequent stage of the sequential data process, where each of the plurality of memory units is sized based on relative locations of the stage providing the intermediate data set and the at least one subsequent stage in the sequential data process.

Systems, methods and devices for a multistage sequential data process

The present disclosure is drawn to the reduction of parameters in fully connected layers of neural networks. For a layer whose output is defined by y=Wx, where y&#x2208;Ris the output vector, x&#x2208;Ris the input vector, and W&#x2208;Ris a matrix of connection parameters, matrices Uand Vare defined and submatrices Ware computed as the product of Uand V, so that W=VU, and W is obtained by appending submatrices W.

Reduction of parameters in fully connected layers of neural networks by low rank factorizations

A method for training a neural network comprising at least one layer comprising a plurality of input nodes, a plurality of output nodes, and a plurality of connections for connecting each one of the plurality of input nodes to each one of the plurality of output nodes, is provided. The method comprises pseudo-randomly selecting a subset of the plurality of connections, each connection of the plurality of connections having associated therewith a weight parameter and a probability of being retained in the neural network, generating output data by feeding input data over the subset of connections, computing an error between the generated output data and desired output data, and, for at least one connection in the subset of connections, determining a contribution of the weight parameter to the error and updating the probability of being retained in the neural network accordingly.

System and method for training a neural network

A memory control unit for handling data stored in a memory device includes a first interface to an interconnection with at least one memory bank; a second interface for communicating with a data requesting unit; and a memory quantization unit. The memory quantization unit is configured to: obtain, via the first interface, a first weight value from the at least one memory bank; quantize the first weight value to generate at least one quantized weight value having a shorter bit length than a bit length of the first weight value; and communicate the at least one quantized weight value to the data requesting unit via the second interface.

Systems, methods and devices for data quantization

The present disclosure is drawn to the reduction of parameters in fully connected layers of neural networks. For a layer whose output is defined by y=Wx, where y is the output vector, x is the input vector, and W is a matrix of connection parameters, vectors uand vare defined and submatrices Ware computed as the outer product of uand v, so that W=v&#x2297;u, and W is obtained by appending submatrices W.

Growing community of inventors

Markham, Canada

Vanessa Courville