Alibaba Group Holding Limited

Alibaba Damo (hangzhou) Technology Co., Ltd.

Yuhao Wang

Hongzhong Zheng

Dimin Niu

Wei Han

Fei Sun

Minghai Qin

Fei Xue

Lide Duan

Shuangchen Li

Yuan Xie

Yen-Kuang Chen

Xiaoxin Fan

Yen-kuang Chen

Kai Xu

Shengcheng Wang

Yijin Guan

Tao Zhang

Jian Lou

Yuanwei Fang

Chunsheng Liu

Zhibin Xiao

Ao Ren

The maximum capacity of a very fast memory in a system that requires very fast memory access times is increased by adding a memory with remote access times that are slower than required, and then moving infrequently accessed data from the memory with the very fast access times to the memory with the slow access times.

Processing system that increases the capacity of a very fast memory

The total memory space that is logically available to a processor in a general-purpose graphics processing unit (GPGPU) module is increased to accommodate terabyte-sized amounts of data by utilizing the memory space in an external memory module, and by further utilizing a portion of the memory space in a number of other external memory modules.

Processing system that increases the memory capacity of a GPGPU

An input weight pattern of a machine learning model may be received. The input weight pattern may be pruned to produce an output weight pattern based on a predetermined pruning algorithm. The pruning algorithm may include partitioning the input weight pattern into a plurality of sub-patterns, each row of the input weight pattern including sub-rows of a first number of sub-patterns, and each column of the input weight pattern including sub-columns of a second number of sub-patterns; and pruning sub-columns and sub-rows from the plurality of sub-patterns to achieve predetermined column and row sparsities respectively, with a constraint that at least one sub-row in each row of the input weight pattern is not pruned. The output weight pattern may further be compressed to produce a compact weight pattern. The compact weight pattern has lower memory and computational overheads as compared to the input weight pattern for the machine learning model.

Structured pruning for machine learning model

A system-in-package architecture in accordance with aspects includes a logic die and one or more memory dice coupled together in a three-dimensional slack. The logic die can include one or more global building blocks and a plurality of local building blocks. The number of local building blocks can be scalable. The local building blocks can include a plurality of engines and memory controllers. The memory controllers can be configured to directly couple one or more of the engines to the one or more memory dice. The number and type of local building blocks, and the number and types of engines and memory controllers can be scalable.

Scalable system-in-package architectures

A dual-model memory interface of a computing system is provided, configurable to present memory interfaces having differently-graded bandwidth capacity to different processors of the computing system. A mode switch controller of the memory interface controller, based on at least an arbitration rule written to a configuration register, switches the memory interface controller between a narrow-band mode and a wide-band mode. In each mode, the memory interface controller disables either a plurality of narrow-band memory interfaces of the memory interface controller according to a first bus standard, or a wide-band memory interface of the memory interface controller according to a second bus standard. The memory interface controller virtualizes a plurality of system memory units of the computing system as a virtual wide-band memory unit according to the second bus standard, or virtualizes a system memory unit of the computing system as a virtual narrow-band memory unit according to the first bus standard.

Dual-modal memory interface controller

The present disclosure relates to a computer-implemented method for processing video data. The method comprises receiving a user input corresponding to a first picture of the video data, generating, based on the user input, prediction information of the first picture with respect a reference picture of the video data, and encoding the first picture using the prediction information.

Video encoding technique utilizing user guided information in cloud environment

The present disclosure relates to an accelerator for systolic array-friendly data placement. The accelerator may include: a systolic array comprising a plurality of operation units, wherein the systolic array is configured to receive staged input data and perform operations using the staged input to generate staged output data, the staged output data comprising a number of segments; a controller configured to execute one or more instructions to generate a pattern generation signal; a data mask generator; and a memory configured to store the staged output data using the generated masks. The data mask generator may include circuitry configured to: receive the pattern generation signal from the controller, and, based on the received signal, generate a mask corresponding to each segment of the staged output data.

Systolic array-friendly data placement and control based on masked write

A chip or integrated circuit includes a layer that includes a first device and a second device. A scribe line is located between the first device and the second device and separates the first device from the second device. An electrically conductive connection traverses the scribe line and is coupled to the first device and the second device, thus connecting the first and second devices.

Using electrical connections that traverse scribe lines to connect devices on a chip

The present disclosure relates to a method for controlling execution of a GEMM operation on an accelerator comprising multiple computation units, a first memory device, and a second memory device. The method comprises determining an execution manner of the GEMM operation, the execution manner comprising partition information of the GEMM operation and computation unit allocation information of the partitioned GEMM operation; generating one or more instructions to compute the partitioned GEMM operation on one or more allocated computation units; and issuing the one or more instructions to at least one of a first queue and a second queue, which enables at least one of a first local controller and a second local controller to execute the one or more instructions, wherein the first local controller and the second local controller are configured to control data movement between the computation units, the first memory device, and the second memory device.

Programmable and hierarchical control of execution of GEMM operation on accelerator

A system for processing encoded image components for artificial intelligence tasks. The system can include one or more compute units, one or more controllers and memory. The one or more controllers can include one or more micro-op schedulers and one or more channel switches. The one or more compute units can be configured to process components of the transformed domain image data according to one or more micro-operations for an artificial intelligence task. The one or more channel switches can be configured to selectively control the transfer of the components of transformed domain image data to the one or more compute units based on one or more gating flags. The one or more channel switches can also be configured to selectively control generation of the one or more micro-operations by the one or more micro-op schedulers based on the one or more gating flags.

Techniques to dynamically gate encoded image components for artificial intelligence tasks

A method of scheduling instructions in a processing system comprising a processing unit and one or more co-processors comprises dispatching a plurality of instructions from a master processor to a co-processor of the one or more co-processors, wherein each instruction of the plurality of instructions comprises one or more additional fields, wherein at least one field comprises grouping information operable to consolidate the plurality of instructions for decomposition, and wherein at least one field comprises control information. The method also comprises decomposing the plurality of instructions into a plurality of fine-grained instructions, wherein the control information comprises rules associated with decomposing the plurality of instructions into the plurality of fine-grained instructions. Further, the method comprises scheduling the plurality of fine-grained instructions to execute on the co-processor, wherein the scheduling is performed in a non-sequential order.

Methods of breaking down coarse-grained tasks for fine-grained task re-scheduling

Image data is accessed. The image data includes frequency domain components. A subset of the frequency domain components is selected based on the relative importance of the frequency domain components. Only the subset of the frequency domain components is provided to an accelerator that executes a neural network to perform an artificial intelligence task using the subset of frequency domain components.

Using selected components of frequency domain image data in artificial intelligence tasks

The systems and methods are configured to efficiently and effectively include processing capabilities in memory. In one embodiment, a processing in memory (PIM) chip a memory array, logic components, and an interconnection network. The memory array is configured to store information. In one exemplary implementation the memory array includes storage cells and array periphery components. The logic components can be configured to process information stored in the memory array. The interconnection network is configured to communicatively couple the logic components. The interconnection network can include interconnect wires, and a portion of the interconnect wires are located in a metal layer area that is located above the memory array.

Memory interconnection architecture systems and methods

A system for determining the importance of encoded image components for artificial intelligence tasks includes an image capture or storage unit, a processor and a communication interface. The processor can receive components of transformed domain image data from the one or more image capture or storage units across the communication interface. The processor can be configured to determine the relative importance of the components of the transformed domain image data for an artificial intelligence task.

Techniques for determining importance of encoded image components for artificial intelligence tasks

Methods and systems are provided for improving memory control. A memory architecture includes a plurality of memory units and an interface. A respective memory unit of the plurality of memory units is configured with a Processing-In-Memory (PIM) architecture. The interface includes a plurality of lines. The interface is coupled between the plurality of memory units and a host. The interface is configured to receive one or more signals from a host via the plurality of lines. The respective memory unit of the plurality of memory units is coupled with a respective line of the plurality of lines, and the respective memory unit is further configured to receive a respective signal of the one or more signals via the interface so as to be individually selected by the host.

Growing community of inventors

Hangzhou, China

Yuhao Wang