Alibaba Group Holding Limited

University of Michigan

Yuan Xie

Tao Zhang

Shuangchen Li

Yuhao Wang

Weifeng Zhang

Guoyang Chen

Zhenyu Gu

Yu Pu

Hongzhong Zheng

Dimin Niu

Fei Sun

Yen-kuang Chen

Minghai Qin

Clark T-c Nguyen

Kai Xu

Jilan Lin

Yongzhi Zhang

Clark T -c Nguyen

Chunsheng Liu

Zhibin Xiao

Xiaoxin Fan

Pengcheng Li

Yongbin Gu

Chao Cheng

Yu-Wei Lin

Sheng-Shian Li

Maohua Zhu

Liu Liu

Ao Ren

Methods and systems are provided for implementing training of learning models, including obtaining a pre-trained weight set for a learning model on a sample dataset and on a first loss function; selecting at least two tasks having heterogeneous features to be computed by a reference model; obtaining a reference dataset for the at least two tasks; designating a second loss function for feature embedding between the heterogeneous features of the at least two tasks; training the learning model on the first loss function and training the reference model on the second loss function, in turn; and updating the weight set based on a feature embedding learned by the learning model and a feature embedding learned by the reference model, in turn. Methods and systems of the present disclosure may alleviate computational overhead incurred by executing the learning model and loading different weight sets at a central network of the model.

Multi-task segmented learning models

The present disclosure relates to systems and methods for providing vector-wise sparsity in neural networks. In some embodiments, an exemplary method for providing vector-wise sparsity in a neural network, comprises: dividing a matrix associated with the neural network into a plurality of vectors; selecting a first subset of non-zero elements from the plurality of vectors to form a pruned matrix; and outputting the pruned matrix for executing the neural network using the pruned matrix.

Systems and methods for providing vector-wise sparsity in a neural network

Embodiments of the disclosure provide methods and systems for processing a neural network associated with an input matrix having a first number of elements. The method can include: dividing the input matrix into a plurality of vectors, each vector having a second number of elements; grouping the plurality of vectors into a first group of vectors and a second group of vectors; and pruning the first group of vectors and the second group of vectors.

Method and system for processing a neural network

The present disclosure provides methods, systems, and non-transitory computer readable media for managing a primary storage unit of an accelerator. The methods include assessing activity of the accelerator; assigning, based on the assessed activity of the accelerator, a lease to a group of one or more pages of data on the primary storage unit, wherein the assigned lease indicates a lease duration; and marking, in response to the expiration of the lease duration indicated by the lease, the group of one or more pages of data as an eviction candidate.

Methods and systems for managing an accelerator's primary storage unit

Methods and devices, the method including receiving a matrix of a neural network model; classifying at least a portion of the matrix as a first section based on a first distribution pattern of non-zero elements of the portion of the matrix; and identifying memory addresses of the non-zero elements in the first section of the matrix for loading, according to a first order determined based on the first distribution pattern, the non-zero elements in the first section into one or more vector registers.

Method and device for matrix multiplication optimization using vector registers

Methods and apparatus for reducing a size of a neural network model, the method including: compressing data of the neural network model; identifying structure information of a vector register, wherein the structure information includes a number of registers included in the vector register; comparing a number of elements in the compressed data with a first condition, wherein the first condition is determined based on the number of registers in the vector register; and in response to the number of elements satisfying the first condition, associating the compressed data with the vector register to enable loading the compressed data to the vector register.

Method and device for reducing a size of a neural network model

Systems and methods for dynamically executing sparse neural networks are provided. In one implementation, a system for providing dynamic sparsity in a neural network may include at least one memory storing instructions and at least one processor configured to execute the instructions to: reduce an input vector and a set of weights of the neural network, execute an input layer of the neural network using the reduced input vector and set of weights to generate a reduced output vector; expand the reduced output vector to a full output vector using first predictable output neurons (PONs); using a PON map, reduce a dimension of the full output vector; execute subsequent layers of the neural network using the reduced full output vector to produce a second reduced output vector; and expand the second reduced output vector to a second full output vector using second PONs.

Systems and methods for accelerating sparse neural network execution

An input weight pattern of a machine learning model may be received. The input weight pattern may be pruned to produce an output weight pattern based on a predetermined pruning algorithm. The pruning algorithm may include partitioning the input weight pattern into a plurality of sub-patterns, each row of the input weight pattern including sub-rows of a first number of sub-patterns, and each column of the input weight pattern including sub-columns of a second number of sub-patterns; and pruning sub-columns and sub-rows from the plurality of sub-patterns to achieve predetermined column and row sparsities respectively, with a constraint that at least one sub-row in each row of the input weight pattern is not pruned. The output weight pattern may further be compressed to produce a compact weight pattern. The compact weight pattern has lower memory and computational overheads as compared to the input weight pattern for the machine learning model.

Structured pruning for machine learning model

The present disclosure relates to systems and methods for providing block-wise sparsity in neural networks. In one implementation, a system for providing block-wise sparsity in a neural network may include at least one memory storing instructions and at least one processor configured to execute the instructions to: divide a matrix of weights associated with a neural network into a plurality of blocks; extract non-zero elements from one or more of the plurality of blocks; re-encode the extracted non-zero elements as vectors with associated coordinates of the extracted non-zero elements within the one or more blocks; enforce input sparsity in the neural network corresponding to the associated coordinates; and execute the neural network using the vectors and the enforced input sparsity.

Systems and methods for providing block-wise sparsity in a neural network

The systems and methods are configured to efficiently and effectively access memory. In one embodiment, a memory controller comprises a request queue, a buffer, a control component, and a data path system. The request queue receives memory access requests. The control component is configured to process information associated with access requests via a first narrow memory channel and a second narrow memory channel. The first narrow memory channel and the second narrow memory channel can have a portion of command/control communication lines and address communication lines that are included in and shared between the first narrow memory channel and the second narrow memory channel. The data path system can include a first data module and one set of unshared data lines associated with the first memory channel and a second data module and another set of unshared data lines associated with second memory channel.

Narrow DRAM channel systems and methods

A processing element/unit can include a plurality of networks, a plurality of cores, crossbar interconnects, a plurality of memory controllers and local memory on an integrated circuit (IC) chip. The plurality of cores can be coupled together by the plurality of networks on chip. The crossbar interconnects can couple the networks of cores to the plurality of memory controllers. The plurality of memory controllers can be configured to access data stored in off-chip memory. The local memory can be configured to cache portions of the accessed data. The local memory can be directly accessible by the network of processing cores, or can be distributed across the plurality of memory controllers. The memory controllers can be narrow channel (NC) memory controllers having widths of 4, 8, 12, 16 or a multiple of 4 bits.

Processing accelerator architectures

A system for processing encoded image components for artificial intelligence tasks. The system can include one or more compute units, one or more controllers and memory. The one or more controllers can include one or more micro-op schedulers and one or more channel switches. The one or more compute units can be configured to process components of the transformed domain image data according to one or more micro-operations for an artificial intelligence task. The one or more channel switches can be configured to selectively control the transfer of the components of transformed domain image data to the one or more compute units based on one or more gating flags. The one or more channel switches can also be configured to selectively control generation of the one or more micro-operations by the one or more micro-op schedulers based on the one or more gating flags.

Techniques to dynamically gate encoded image components for artificial intelligence tasks

A method and an apparatus for scheduling commands in a virtual computing environment includes picking a command. It is determined whether the command is a synchronization command or a conditional command. A synchronization command is an independent command. A conditional command is a dependent command that depends on a synchronization command. In response to the command being determined as the synchronization command, a waiting queue is enabled for the command, the waiting queue storing conditional commands dependent on a running synchronization command. The command is dispatched to a processing engine.

Scheduling commands in a virtual computing environment

A system for determining the importance of encoded image components for artificial intelligence tasks includes an image capture or storage unit, a processor and a communication interface. The processor can receive components of transformed domain image data from the one or more image capture or storage units across the communication interface. The processor can be configured to determine the relative importance of the components of the transformed domain image data for an artificial intelligence task.

Growing community of inventors

Sunnyvale, CA, United States of America

Yuan Xie