Samsung Electronics Co., Ltd.

Alibaba Group Holding Limited

Alibaba (china) Co., Ltd.

Alibaba Damo (hangzhou) Technology Co., Ltd.

Dimin Niu

Hongzhong Zheng

Mu-Tien Chang

Shuangchen Li

Krishna Teja Malladi

Jangseok Choi

Sun Young Lim

Indong Kim

Tianchan Guan

Hyun-Joong Kim

Won-hyung Song

Lide Duan

Wei Han

Yuhao Wang

Craig Hanson

Yijin Guan

Chaohong Hu

Dongyan Jiang

Peng Gu

Fei Xue

Sun-Young Lim

Bob Brennan

Xiaoxin Fan

Frederic Sala

Shaungchen Li

Prasun Gera

Jilan Lin

Youngjin Cho

Fei Sun

Jae-Gon Lee

Yuan Xie

Heehyun Nam

Shengcheng Wang

Youngsik Kim

Heng Liu

Zhe Zhang

Kyung-Chang Ryoo

Weifeng Zhang

Yuanwei Fang

Guoyang Chen

Yawen Zhang

Chunsheng Liu

In-Su Choi

Zhibin Xiao

Suhas

Hongyu Liu

Yingmin Li

Mu Tien Chang

Shuangcheng Li

In-Dong Kim

Xipeng Shen

Jingjun Chu

Honzhong Zheng

A high-bandwidth memory (HBM) includes a memory and a controller. The controller receives a data write request from a processor external to the HBM and the controller stores an entry in the memory indicating at least one address of data of the data write request and generates an indication that a data bus is available for an operation during a cycle time of the data write request based on the data write request comprising sparse data or data-value similarity. Sparse data includes a predetermined percentage of data values equal to zero, and data-value similarity includes a predetermined amount of spatial value locality of the data values. The predetermined percentage of data values equal to zero of sparse data and the predetermined amount of spatial value locality of the special-value pattern are both based on a predetermined data granularity.

High bandwidth memory system

The present disclosure discloses a processor. The processor is used to perform parallel computation and includes a logic die and a memory die. The logic die includes a plurality of processor cores and a plurality of networks on chip, wherein each processor core is programmable. The plurality of networks on chip are correspondingly connected to the plurality of processor cores, so that the plurality of processor cores form a two-dimensional mesh network. The memory die and the processor core are stacked vertically, wherein the memory die includes a plurality of memory tiles, and when the processor performs the parallel computation, the plurality of memory tiles do not have cache coherency; wherein, the plurality of memory tiles correspond to the plurality of processor cores in a one-to-one or one-to-many manner.

Two-dimensional processing array with a vertically stacked memory tile array

A processor includes a plurality of memory units, each of the memory units including a plurality of memory cells, wherein each of the memory units is configurable to operate as memory, as a computation unit, or as a hybrid memory-computation unit.

Scaling out architecture for dram-based processing unit (DPU)

This application describes systems and methods for facilitating memory access for graph neural network (GNN) processing. An example method includes fetching, by an access engine circuitry implemented on a circuitry board, a portion of structure data of a graph from a pinned memory in a host memory of a host via a first peripheral component interconnect express (PCIe) connection; performing node sampling using the fetched portion of the structure data of the graph to select one or more sampled nodes; fetching, by the access engine circuitry, a portion of attribute data of the graph from the pinned memory via the first PCIe connection; sending the fetched portion of the attribute data of the graph to one or more processors; and performing, by the one or more processors, GNN processing for the graph using the fetched portion of the attribute data of the graph.

Graph acceleration solution with cloud FPGA

The present application discloses a computing system for implementing an artificial neural network model. The artificial neural network model has a structure of multiple layers. The computing system comprises a first processing unit, a second processing unit, and a third processing unit. The first processing unit performs computations of the first layer based on a first part of input data of the first layer to generate a first part of output data. The second processing unit performs computations of the first layer based on a second part of the input data of the first layer so as to generate a second part of the output data. The third processing unit performs computations of the second layer based on the first part and the second part of the output data. The first processing unit, the second processing unit, and the third processing unit have the same structure.

Computing system for implementing artificial neural network models and method for implementing artificial neural network models

A high bandwidth memory system. In some embodiments, the system includes: a memory stack having a plurality of memory dies and eight 128-bit channels; and a logic die, the memory dies being stacked on, and connected to, the logic die; wherein the logic die may be configured to operate a first channel of the 128-bit channels in: a first mode, in which a first 64 bits operate in pseudo-channel mode, and a second 64 bits operate as two 32-bit fine-grain channels, or a second mode, in which the first 64 bits operate as two 32-bit fine-grain channels, and the second 64 bits operate as two 32-bit fine-grain channels.

Bandwidth boosted stacked memory

A method of correcting a memory error of a dynamic random-access memory module (DRAM) using a double data rate (DDR) interface, the method includes conducting a memory transaction including multiple bursts with a memory controller to send data from data chips of the DRAM to the memory controller, detecting one or more errors using an ECC chip of the DRAM, determining a number of the bursts having the errors using the ECC chip of the DRAM, determining whether the number of the bursts having the errors is greater than a threshold number, determining a type of the errors, and directing the memory controller based on the determined type of the errors, wherein the DRAM includes a single ECC chip per memory channel.

DRAM assist error correction mechanism for DDR SDRAM interface

The present application provides a computer system, a memory expansion device and a method for use in the computer system. The computer system includes multiple hosts and multiple memory expansion devices; the memory expansion devices correspond to the hosts in a one-to-one manner. Each host includes a CPU and a memory; each memory expansion device includes a first interface and multiple second interfaces. The first interface is configured to allow each memory expansion device to communicate with the corresponding CPU via a first coherence interconnection protocol, and the second interface is configured to allow each memory expansion device to communicate with a portion of memory expansion devices via a second coherence interconnection protocol. Any two memory expansion devices communicate with each other via at least two different paths, and the number of memory expansion devices that at least one of the two paths passes through is not more than one.

Computer system, memory expansion device and method for use in computer system

A system-in-package including a logic die and one or more memory dice can include a reliability availability serviceability (RAS) memory management unit (MMU) for memory error detection, memory error prediction and memory error handling. The RAS MMU can receive memory health information, on-die memory error information, system error information and read address information for the one or more memory dice. The RAS MMU can manage the memory blocks of the one or more memory dice based on the memory health information, on-die memory error type, system error type and read address. The RAS MMU can also further manage the memory blocks based on received on-die memory temperature information and or system temperature information.

Error detection, prediction and handling techniques for system-in-package memory architectures

The presented systems enable efficient and effective network communications. The presented systems enable efficient and effective network communications. In one embodiment a memory device includes a memory module, including a plurality of memory chips configured to store information; and an inter-chip network (ICN)/shared smart memory extension (SMX) memory interface controller (ICN/SMX memory interface controller) configured to interface between the memory module and an inter-chip network (ICN), wherein the ICN is configured to communicatively couple the memory device to a parallel processing unit (PPU). In one exemplary implementation, the ICN/SMX memory controller includes a plurality of package buffers, an ICN physical layer interface, a PRC/MAC interface, and a switch. The memory device and be a memory card including memory module (e.g., DDR DIMM, etc.).

Flexible memory extension systems and methods

A 3D-stacked memory device including: a base die including a plurality of switches to direct data flow and a plurality of arithmetic logic units (ALUs) to compute data; a plurality of memory dies stacked on the base die; and an interface to transfer signals to control the base die.

3D-stacked memory with reconfigurable compute logic

A memory module that includes a non-volatile memory and an asynchronous memory interface to interface with a memory controller is presented. The asynchronous memory interface may use repurposed pins of a double data rate (DDR) memory channel to send an asynchronous data to the memory controller. The asynchronous data may be device feedback indicating a status of the non-volatile memory.

Asynchronous communication protocol compatible with synchronous DDR protocol

A memory management method of a data processing system is provided. The memory management method includes: creating a first memory zone and a second memory zone related to a first node of a first server, wherein the first server is located in the data processing system, and the first node includes a processor and a first memory; mapping the first memory zone to the first memory; and mapping the second memory zone to a second memory of a second server, wherein the second server is located in the data processing system, and the processor is configured to access the second memory of the second server through an interface circuit of the first server and through an interface circuit of the second server.

Data processing system and memory management method of data processing system

According to one general aspect, an apparatus may include a plurality of stacked integrated circuit dies that include a memory cell die and a logic die. The memory cell die may be configured to store data at a memory address. The logic die may include an interface to the stacked integrated circuit dies and configured to communicate memory accesses between the memory cell die and at least one external device. The logic die may include a reliability circuit configured to ameliorate data errors within the memory cell die. The reliability circuit may include a spare memory configured to store data, and an address table configured to map a memory address associated with an error to the spare memory. The reliability circuit may be configured to determine if the memory access is associated with an error, and if so completing the memory access with the spare memory.

HBM RAS cache architecture

A general matrix-matrix multiplication (GEMM) dataflow accelerator circuit is disclosed that includes a smart 3D stacking DRAM architecture. The accelerator circuit includes a memory bank, a peripheral lookup table stored in the memory bank, and a first vector buffer to store a first vector that is used as a row address into the lookup table. The circuit includes a second vector buffer to store a second vector that is used as a column address into the lookup table, and lookup table buffers to receive and store lookup table entries from the lookup table. The circuit further includes adders to sum the first product and a second product, and an output buffer to store the sum. The lookup table buffers determine a product of the first vector and the second vector without performing a multiply operation. The embodiments include a hierarchical lookup architecture to reduce latency. Accumulation results are propagated in a systolic manner.

Dataflow accelerator architecture for general matrix-matrix multiplication and tensor computation in deep learning

Growing community of inventors

Sunnyvale, CA, United States of America

Dimin Niu