Alibaba (China) Co., Ltd.

Alibaba Group Holding Limited

T-Head (Shanghai) Semiconductor Co., Ltd.

Fei Sun

Haoran Li

Yuan Gao

Ruiguang Zhong

Hongzhong Zheng

Lide Duan

Fei Xue

Chen Zhang

Yijin Guan

Ling Liang

Guyue Huang

Yangjie Zhou

This application describes an accelerator, a computer system, and a method for adaptive graph-to-stream scheduling in processors. An example method may include receiving a computation graph for a GPU, the computation graph comprising (1) a plurality of nodes representing a plurality of kernels for the GPU to execute and (2) a plurality of edges representing execution dependencies among the plurality of kernels; performing one or more wave partitions on the computation graph to determine a plurality of waves of kernels; obtaining a kernel resource table comprising resource usage of each kernel; mapping the plurality of kernels into a plurality of streams based on the plurality of waves and the kernel resource table; and executing the plurality of streams on the GPU, wherein kernels mapped in a same stream are executed sequentially by the GPU, and kernels mapped to different streams are concurrently executable by the GPU.

System and method for adaptive graph-to-stream scheduling

A GPU and a GPU method are provided. The GPU includes stream multiprocessors, an available hardware resource table, resource comparator, stream scheduler, and global dispatcher. Each stream multiprocessor executes at least one thread block. The available hardware resource table records hardware resources available to the stream multiprocessors. The resource comparator selects from first-priority kernel codes in the kernel streams at least one first dispatchable kernel code whose required hardware resources are less than hardware resources available to the stream multiprocessors according to the available hardware resource table. The stream scheduler selects from the at least one first dispatchable kernel code a kernel code as a selected kernel code. The global dispatcher dispatches thread blocks in the selected kernel code to the stream multiprocessors for execution and updates the available hardware resource table according to usage conditions of hardware resources of the stream multiprocessors.

GPU and GPU method

Zero skipping sparsity techniques for reduced data movement between memory and accelerators and reduced computational workload of accelerators. The techniques include detection of zero and near-zero values on the memory. The non-zero values are transferred to the accelerator for computation. The zero and near-zero values are written back within the memory as zero values.

Zero skipping techniques for reducing data movement

The present application discloses a warp execution method used for SPs of an SM of a GPU and an associated GPU. The SPs share a scratchpad memory, and the warp execution method includes: when the predetermined time point for warp-loading is reached, checking a first indicator to obtain a size of a space with the status of blank in the scratchpad memory, to determining whether to load the warp, wherein the first indicator is used to indicate a starting position of a space with the status of data-in-use and an ending position of the space with the status of blank; and when the predetermined time point for computing is reached, checking a second indicator and a third indicator to obtain a size of a space with the status of data-not-in-use in the scratchpad memory, to determining whether to compute the warp.

Warp execution method and associated GPU

A processor for sparse matrix calculation includes an on-chip memory, a cache, a gather/scatter engine, and a core. The on-chip memory stores a first matrix or vector, and the cache stores a compressed sparse second matrix data structure. The compressed sparse second matrix data structure includes a value array including non-zero element values of the sparse second matrix, where each entry includes a given number of element values; and a column index array where each entry includes the given number of offsets matching the value array. The gather/scatter engine gathers element values of the first matrix or vector using the column index array of the sparse second matrix. In a hybrid horizontal/vertical implementation, the gather/scatter engine gathers sets of element values from sets of rows and from different sub-banks within the same rows based on the column index array of the sparse matrix.

Sparse matrix calculations utilizing tightly coupled memory and gather/scatter engine

The present disclosure provides a power management method, a multi-processing unit system, and a power management module thereof. The multi-processing unit system comprises a plurality of local power management units and a global power management unit, each of the local power management units corresponds to a processing unit of the multi-processing unit system. The power management method comprises: obtaining, using the global power management unit, a global power budget for the multi-processing unit system; allocating, using the global power management unit, local power budget for each of the local power management units according to the global power budget and power management parameters of the processing units; managing, using the local power management unit, local power resources of corresponding processing unit based on the allocated local power budget; reporting, using the local power management unit, the power management parameters of the processing unit to the global power management unit. The present disclosure improves the overall power efficiency for the multi-processing unit system.

Power management method, multi-processing unit system and power management module

An instruction processing apparatus is disclosed. The instruction processing apparatus includes: a selector is configured to parse out a command type and a buffer identifier from a command, provide received data and the buffer identifier to a parser if the command type is configuration, and provide the received data and the buffer identifier to an operation circuit if the command type is execution; the parser is configured to parse out an instruction sequence from the data, store the instruction sequence into an instruction buffer corresponding to the buffer identifier, and store an operand of each instruction into a register file; and the operation circuit is configured to drive the instruction buffer to execute each instruction and generate a control signal, and trigger a plurality of execution units to perform operations based on received control signals and operands. The apparatus may be dedicated to processing various neural network applications.

Growing community of inventors

San Jose, CA, United States of America

Fei Sun