Samsung Electronics Co., Ltd.

Rambus Inc.

Alibaba Group Holding Limited

Alibaba (china) Co., Ltd.

T-head (shanghai) Semiconductor Co., Ltd.

Intel Corporation

Alibaba Damo (hangzhou) Technology Co., Ltd.

Hongzhong Zheng

Dimin Niu

Krishna Teja Malladi

Mu-Tien Chang

Shuangchen Li

Jangseok Choi

Dongyan Jiang

Chaohong Hu

Sun Young Lim

Indong Kim

Peng Gu

Trung Am Diep

Tianchan Guan

Brent Steven Haukness

Lide Duan

Fei Xue

Hyun-Joong Kim

Won-hyung Song

Jongmin Gim

Frederick Abbott Ware

Yuhao Wang

Wei Han

Robert Brennan

Uksong Kang

Craig Hanson

Yijin Guan

Qiang Peng

Wonseok Lee

Liang Yin

Keith Chan

Bob Brennan

Suhas

Tackhwi Lee

Fei Sun

Andrew Zhenwen Chang

Heng Liu

Shengcheng Wang

James Tringali

Frederic Sala

Shaungchen Li

Mingyu Gao

Jilan Lin

Prasun Gera

Changhui Lin

Youngjin Cho

Eugene Gorbatov

Yen-kuang Chen

Ulf Hanebutte

Howard S David

Zhan Ping

Yuan Xie

Hongyu Liu

Heehyun Nam

Joseph James Tringali

Mu Tien Chang

Ian P Shaeffer

John Eric Linstadt

Youngsik Kim

Jae-Gon Lee

Zhe Zhang

Wentao Wu

Sun-Young Lim

J James Tringali

Jinhyun Kim

Kyung-Chang Ryoo

Xiang Gao

Vidyabhushan Mohan

Yen-Kuang Chen

Feng Zhu

Weifeng Zhang

Yuanwei Fang

Guoyang Chen

Yawen Zhang

Jifeng Wang

Xiaoxin Fan

Jiajing Jin

Yuming Xu

Jiu Heng

Hyungseuk Kim

Rakesh Ramesh

Yingmin Li

Qichen Zhang

Tyler Stocksdale

Prashant Jayaprakash Nair

Jingjun Chu

Mehmet Günhan Ertosun

Shuangcheng Li

Xipeng Shen

Yangjie Zhou

Bowen Huang

A high-bandwidth memory (HBM) includes a memory and a controller. The controller receives a data write request from a processor external to the HBM and the controller stores an entry in the memory indicating at least one address of data of the data write request and generates an indication that a data bus is available for an operation during a cycle time of the data write request based on the data write request comprising sparse data or data-value similarity. Sparse data includes a predetermined percentage of data values equal to zero, and data-value similarity includes a predetermined amount of spatial value locality of the data values. The predetermined percentage of data values equal to zero of sparse data and the predetermined amount of spatial value locality of the special-value pattern are both based on a predetermined data granularity.

High bandwidth memory system

This specification describes methods and systems for accessing attribute data in graph neural network (GNN) processing. An example system includes: a plurality of cores, each of the plurality of cores comprises a key-value fetcher and a filter, and is programmable using a software interface to support a plurality of data formats of the GNN attribute data, wherein: the key-value fetcher is programmable using the software interface to perform key-value fetching associated with accessing the GNN attribute data, and the filter of at least one of the plurality of cores is programmable using the software interface to sample node identifiers associated with accessing the GNN attribute data; and a first memory communicatively coupled with the plurality of cores, wherein the first memory is configured to store data shared by the plurality of cores.

Programmable access engine architecture for graph neural network and graph application

The present disclosure discloses a processor. The processor is used to perform parallel computation and includes a logic die and a memory die. The logic die includes a plurality of processor cores and a plurality of networks on chip, wherein each processor core is programmable. The plurality of networks on chip are correspondingly connected to the plurality of processor cores, so that the plurality of processor cores form a two-dimensional mesh network. The memory die and the processor core are stacked vertically, wherein the memory die includes a plurality of memory tiles, and when the processor performs the parallel computation, the plurality of memory tiles do not have cache coherency; wherein, the plurality of memory tiles correspond to the plurality of processor cores in a one-to-one or one-to-many manner.

Two-dimensional processing array with a vertically stacked memory tile array

A processor includes a plurality of memory units, each of the memory units including a plurality of memory cells, wherein each of the memory units is configurable to operate as memory, as a computation unit, or as a hybrid memory-computation unit.

Scaling out architecture for dram-based processing unit (DPU)

This application describes systems and methods for facilitating memory access for graph neural network (GNN) processing. An example method includes fetching, by an access engine circuitry implemented on a circuitry board, a portion of structure data of a graph from a pinned memory in a host memory of a host via a first peripheral component interconnect express (PCIe) connection; performing node sampling using the fetched portion of the structure data of the graph to select one or more sampled nodes; fetching, by the access engine circuitry, a portion of attribute data of the graph from the pinned memory via the first PCIe connection; sending the fetched portion of the attribute data of the graph to one or more processors; and performing, by the one or more processors, GNN processing for the graph using the fetched portion of the attribute data of the graph.

Graph acceleration solution with cloud FPGA

An SSD includes an MRAM, an NAND memory, and an SSD controller. The SSD controller is configured to receive first data from a host machine, save the first data to an SSD data buffer, fetch the first data from the SSD data buffer and write the first data to the MRAM via the MRAM controller, determine, by the data allocation circuit based on a characteristic of the first data, whether to save the first data to the MRAM or the NAND memory, and in response to determining saving the first data to the NAND memory, read the first data from the MRAM, write the first data to the NAND memory, and erase the first data from the MRAM.

Hybrid solid state drive (SSD) architecture using MRAM and NAND for improved power loss protection and performance

A transaction manager for use with memory is described. The transaction manager can include a write data buffer to store outstanding write requests, a read data multiplexer to select between data read from the memory and the write data buffer, a command queue and a priority queue to store requests for the memory, and a transaction table to track outstanding write requests, each write request associated with a state that is Invalid, Modified, or Forwarded.

Effective transaction table with page bitmap

The present application discloses a computing system for implementing an artificial neural network model. The artificial neural network model has a structure of multiple layers. The computing system comprises a first processing unit, a second processing unit, and a third processing unit. The first processing unit performs computations of the first layer based on a first part of input data of the first layer to generate a first part of output data. The second processing unit performs computations of the first layer based on a second part of the input data of the first layer so as to generate a second part of the output data. The third processing unit performs computations of the second layer based on the first part and the second part of the output data. The first processing unit, the second processing unit, and the third processing unit have the same structure.

Computing system for implementing artificial neural network models and method for implementing artificial neural network models

A high bandwidth memory system. In some embodiments, the system includes: a memory stack having a plurality of memory dies and eight 128-bit channels; and a logic die, the memory dies being stacked on, and connected to, the logic die; wherein the logic die may be configured to operate a first channel of the 128-bit channels in: a first mode, in which a first 64 bits operate in pseudo-channel mode, and a second 64 bits operate as two 32-bit fine-grain channels, or a second mode, in which the first 64 bits operate as two 32-bit fine-grain channels, and the second 64 bits operate as two 32-bit fine-grain channels.

Bandwidth boosted stacked memory

A computer-implemented method for allocating memory bandwidth of multiple CPU cores in a server includes: receiving an access request to a last level cache (LLC) shared by the multiple CPU cores in the server, the access request being sent from a core with a private cache holding copies of frequently accessed data from a memory; determining whether the access request is an LLC hit or an LLC miss; and controlling a memory bandwidth controller based on the determination. The memory bandwidth controller performs a memory bandwidth throttling to control a request rate between the private cache and the last level cache. The LLC hit of the access request causes the memory bandwidth throttling initiated by the memory bandwidth controller to be disabled and the LLC miss of the access request causes the memory bandwidth throttling initiated by the memory bandwidth controller to be enabled.

Systems and methods for memory bandwidth allocation

A method of correcting a memory error of a dynamic random-access memory module (DRAM) using a double data rate (DDR) interface, the method includes conducting a memory transaction including multiple bursts with a memory controller to send data from data chips of the DRAM to the memory controller, detecting one or more errors using an ECC chip of the DRAM, determining a number of the bursts having the errors using the ECC chip of the DRAM, determining whether the number of the bursts having the errors is greater than a threshold number, determining a type of the errors, and directing the memory controller based on the determined type of the errors, wherein the DRAM includes a single ECC chip per memory channel.

DRAM assist error correction mechanism for DDR SDRAM interface

A cache memory includes cache lines to store information. The stored information is associated with physical addresses that include first, second, and third distinct portions. The cache lines are indexed by the second portions of respective physical addresses associated with the stored information. The cache memory also includes one or more tables, each of which includes respective table entries that are indexed by the first portions of the respective physical addresses. The respective table entries in each of the one or more tables are to store indications of the second portions of respective physical addresses associated with the stored information.

Methods and apparatuses for addressing memory caches

The present application provides a computer system, a memory expansion device and a method for use in the computer system. The computer system includes multiple hosts and multiple memory expansion devices; the memory expansion devices correspond to the hosts in a one-to-one manner. Each host includes a CPU and a memory; each memory expansion device includes a first interface and multiple second interfaces. The first interface is configured to allow each memory expansion device to communicate with the corresponding CPU via a first coherence interconnection protocol, and the second interface is configured to allow each memory expansion device to communicate with a portion of memory expansion devices via a second coherence interconnection protocol. Any two memory expansion devices communicate with each other via at least two different paths, and the number of memory expansion devices that at least one of the two paths passes through is not more than one.

Computer system, memory expansion device and method for use in computer system

A system-in-package including a logic die and one or more memory dice can include a reliability availability serviceability (RAS) memory management unit (MMU) for memory error detection, memory error prediction and memory error handling. The RAS MMU can receive memory health information, on-die memory error information, system error information and read address information for the one or more memory dice. The RAS MMU can manage the memory blocks of the one or more memory dice based on the memory health information, on-die memory error type, system error type and read address. The RAS MMU can also further manage the memory blocks based on received on-die memory temperature information and or system temperature information.

Error detection, prediction and handling techniques for system-in-package memory architectures

The presented systems enable efficient and effective network communications. The presented systems enable efficient and effective network communications. In one embodiment a memory device includes a memory module, including a plurality of memory chips configured to store information; and an inter-chip network (ICN)/shared smart memory extension (SMX) memory interface controller (ICN/SMX memory interface controller) configured to interface between the memory module and an inter-chip network (ICN), wherein the ICN is configured to communicatively couple the memory device to a parallel processing unit (PPU). In one exemplary implementation, the ICN/SMX memory controller includes a plurality of package buffers, an ICN physical layer interface, a PRC/MAC interface, and a switch. The memory device and be a memory card including memory module (e.g., DDR DIMM, etc.).

Flexible memory extension systems and methods

Growing community of inventors

Los Gatos, CA, United States of America

Hongzhong Zheng