Alibaba Group Holding Limited

Guoyang Chen

Weifeng Zhang

Yuan Xie

Yu Pu

Yingmin Li

Minxuan Zhou

Yongzhi Zhang

Hongzhong Zheng

Dimin Niu

Fei Sun

Shuangchen Li

Jingjun Chu

Xipeng Shen

Shuai Che

The present disclosure relates to a processing in memory (PIM) enabled device for executing a neural network model. The PIM enabled device comprises a memory block assembly comprising a first array of memory blocks, a second array of memory blocks adjacent to the first array of memory blocks, a plurality of first data links associated with the first array of memory blocks and the second array of memory blocks, wherein each data link of the plurality of first data links communicatively couples two corresponding memory blocks of which are from the first array of memory blocks and the second array of memory blocks respectively, and a second data link communicatively coupled to the plurality of first data links. The data from a first memory block of the first array of memory blocks can be transferable to a second memory block of the second array of memory blocks via the plurality of first data links and the second data link.

Data layout conscious processing in memory architecture for executing neural network model

Methods and devices, the method including receiving a matrix of a neural network model; classifying at least a portion of the matrix as a first section based on a first distribution pattern of non-zero elements of the portion of the matrix; and identifying memory addresses of the non-zero elements in the first section of the matrix for loading, according to a first order determined based on the first distribution pattern, the non-zero elements in the first section into one or more vector registers.

Method and device for matrix multiplication optimization using vector registers

Methods and apparatus for reducing a size of a neural network model, the method including: compressing data of the neural network model; identifying structure information of a vector register, wherein the structure information includes a number of registers included in the vector register; comparing a number of elements in the compressed data with a first condition, wherein the first condition is determined based on the number of registers in the vector register; and in response to the number of elements satisfying the first condition, associating the compressed data with the vector register to enable loading the compressed data to the vector register.

Method and device for reducing a size of a neural network model

The present disclosure relates to a method for scheduling a computation graph on a processing in memory (PIM) enabled device comprising a memory block assembly. The method comprises allocating a first node of the computation graph on a first memory block of a first array of memory blocks in the memory block assembly and allocating a second node of the computation graph on a second memory block of a second array of memory blocks in the memory block assembly, wherein output data of the first node is used for executing the second node. The memory block assembly can be configured to support data transfer from the first memory block to the second memory block via an internal data coupling in the memory block assembly.

Data layout optimization on processing in memory architecture for executing neural network model

Embodiments of the disclosure provide systems and methods for allocating memory space in a memory device. The system can include: a memory device for providing the memory space; and a compiler component configured for: receiving a request for allocating a data array having a plurality of data elements in the memory device, wherein each of the plurality of data elements has a logical address; generating an instruction for allocating memory space for the data array in the memory device based on the request; generating device addresses for the plurality of data elements in the memory device based on logical addresses of the plurality of data elements; and allocating the memory space for the data array in the memory device based on the device addresses and the instruction.

System and method for allocating memory space

The present disclosure relates to computer-implemented systems and methods for scheduling a neural network for execution. In one implementation, a system for scheduling a neural network for execution may include at least one memory storing instructions and at least one processor configured to execute the instructions to determine a profile for one or more applications co-scheduled with at least one neural network; determine a batch size for the at least one neural network based on the determined profile for the one or more applications; and scheduling the one or more applications and the at least one neural network based on the batch size.

Growing community of inventors

San Mateo, CA, United States of America

Guoyang Chen