Advanced Micro Devices Corporation

Johnathan R Alsop

Shaizeen Dilawarhusen Aga

Nuwan S Jayasena

Sergey Blagodurov

Bradford Michael Beckmann

Mohamed Assem Abd ElMohsen Ibrahim

Andrew McCrabb

Yasuko Eckert

Jagadish B Kotra

Sooraj Puthoor

Anthony T Gutierrez

Ganesh Suryanarayan Dasika

Seyedmohammad SeyedzadehDelcheh

Onur Kayiran

Michael Warren Boyer

Alexandru Dutu

Marko Scrbak

Khaled Hamidouche

Kishore Punniyamurthy

Brandon K Potter

Xianwei Zhang

Karthik Ramu Sangaiah

Mahzabeen Islam

Meysam Taassori

Pouya Fotouhi

Laurent S White

Programmable data storage memory hierarchy techniques are described. In one example, a data storage system includes a memory hierarchy and a data movement controller. The memory hierarchy includes a hierarchical arrangement of a plurality of memory buffers. The data movement controller is configured to receive a data movement command and control data movement between the plurality of memory buffers based on the data movement command.

Programmable data storage memory hierarchy

Non-blocking processing system are described. In accordance with the described techniques, a pending range store receives, at a start of a bulk memory operation, a pending memory range of the bulk memory operation. A logic unit includes at least one of check conflict logic or check address logic. The logic unit detects a conflicting memory access based on a target address of the pending memory range conflicting with a memory access request separate from the bulk memory operation. The logic unit performs at least a portion of the bulk memory operation associated with the target address before the memory access request is allowed to proceed.

Non-blocking parallel bulk memory operations

An approach is provided for managing near-memory processing commands ('PIM commands') from multiple processor threads in a manner to prevent interference and maintain correctness at near-memory processing elements. A memory controller uses thread identification information and last command information to issue a PIM command sequence from a first processor thread, directed to a PIM-enabled memory element, while deferring the issuance of PIM command sequences from other processor threads, directed to the same PIM-enabled memory element. After the last PIM command in the PIM command sequence for the first processor thread has been issued, a PIM command sequence for another processor thread is issued, and so on. The approach allows multiple processor threads to concurrently issue fine grained PIM commands to the same PIM-enabled memory element without having to be aware of address-to-memory element mapping, and without having to coordinate with other threads.

Approach for managing near-memory processing commands from multiple processor threads to prevent interference at near-memory processing elements

Systems, apparatuses, and methods for reducing probe filter accesses in response to processing-in-memory (PIM) requests are disclosed. A coherent secondary unit receives PIM requests targeting a corresponding PIM device. For each PIM request that is received, the coherent secondary unit performs a lookup of a PIM address table (PAT). If the address of the PIM request matches an address of an existing entry in the PAT, the coherent secondary unit prevents the PIM request from being sent to a probe filter. Otherwise, if there is no match for the address of the PIM request in the entries of the PAT, the coherent secondary unit sends the PIM request to the probe filter, and the coherent secondary unit creates a new PAT entry for the address of the PIM request. Any subsequent PIM requests to the same address will match with the new entry in the PAT.

Reducing probe filter accesses for processing in memory requests

Methods and apparatuses to control digital data transfer via a memory channel between a memory module and a processor are disclosed. At least one of the memory module or the processor coalesces a plurality of short data words into multicast coalesced block data comprising a single data block for transfer via the memory channel. Each of the plurality of short data words pertains to one of at least two partitioned memory submodules in the memory module. The multicast coalesced block data is communicated over the memory channel.

System and method for coalesced multicast data transfers over memory interfaces

Systems, apparatuses, and methods for dynamically coalescing multi-bank memory commands to improve command throughput are disclosed. A system includes a processor coupled to a memory via a memory controller. The memory also includes processing-in-memory (PIM) elements which are able to perform computations within the memory. The processor generates memory requests targeting the memory which are sent to the memory controller. The memory controller stores commands received from the processor in a queue, and the memory controller determines whether opportunities exist for coalescing multiple commands together into a single multi-bank command. After coalescing multiple commands into a single combined multi-bank command, the memory controller conveys, across the memory bus to multiple separate banks, the single multi-bank command and a multi-bank code specifying which banks are targeted. The memory banks process the command in parallel, and the PIM elements process the data next to each respective bank.

Dynamic multi-bank memory command coalescing

Near-memory compute elements perform memory operations and temporarily store at least a portion of address information for the memory operations in local storage. A broadcast memory command is then issued to the near-memory compute elements that causes the near-memory compute elements to perform a subsequent memory operation using their respective address information stored in the local storage. This allows a single broadcast memory command to be used to perform memory operations across multiple memory elements, such as DRAM banks, using bank-specific address information. In one implementation, the approach is used to process workloads with irregular updates to memory while consuming less command bus bandwidth than conventional approaches. Implementations include using conditional flags to selectively designate address information in local storage that is to be processed with the broadcast memory command.

Approach for performing efficient memory operations using near-memory compute elements

In accordance with the described techniques for bank-level parallelism for processing in memory, a plurality of commands are received for execution by a processing in memory component embedded in a memory. The memory includes a first bank and a second bank. The plurality of commands include a first stream of commands which cause the processing in memory component to perform operations that access the first bank and a second stream of commands which cause the processing in memory component to perform operations that access the second bank. A next row of the first bank that is to be accessed by the processing in memory component is identified. Further, a precharge command is scheduled to close a first row of the first bank and an activate command is scheduled to open the next row of the first bank in parallel with execution of the second stream of commands.

Bank-level parallelism for processing in memory

In accordance with described techniques for filtered responses to memory operation messages, a computing system or computing device includes a memory system that receives messages. A filter component in the memory system receives the responses to the memory operation messages, and filters one or more of the responses based on a filterable condition. A tracking logic component tracks the one or more responses as filtered responses for communication completion.

Filtered responses of memory operation messages

Systems, apparatuses, and methods for determining data placement based on packet metadata are disclosed. A system includes a traffic analyzer that determines data placement across connected devices based on observed values of the metadata fields in actively exchanged packets across a plurality of protocol types. In one implementation, the protocol that is supported by the system is the compute express link (CXL) protocol. The traffic analyzer performs various actions in response to events observed in a packet stream that match items from a pre-configured list. Data movement is handled underneath the software applications by changing the virtual-to-physical address translation once the data movement is completed. After the data movement is finished, threads will pull in the new host physical address into their translation lookaside buffers (TLBs) via a page table walker or via an address translation service (ATS) request.

Data placement with packet metadata

A virtual padding unit provides a virtual padded data structure (e.g., virtually padded matrix) that provides output values for a padded data structure without storing all of the padding elements in memory. When the virtual padding unit receives a virtual memory address of a location in the virtual padded data structure, the virtual padding unit checks whether the location is a non-padded location in the virtual padded data structure or a padded location in the virtual padded data structure. If the location is a padded location in the virtual padded data structure, the virtual padding unit outputs a padding value rather than a value stored in the virtual padded data structure. If the location is a non-padded location in the virtual padded data structure, a value stored at the location is output.

Virtually padding data structures

Arbitrating atomic memory operations, including: receiving, by a media controller, a plurality of atomic memory operations; determining, by an atomics controller associated with the media controller, based on one or more arbitration rules, an ordering for issuing the plurality of atomic memory operations; and issuing the plurality of atomic memory operations to a memory module according to the ordering.

Arbitrating atomic memory operations

Methods and systems are disclosed for performing dataflow execution by an accelerated processing unit (APU). Techniques disclosed include decoding information from one or more dataflow instructions. The decoded information is associated with dataflow execution of a computational task. Techniques disclosed further include configuring, based on the decoded information, dataflow circuitry, and, then, executing the dataflow execution of the computational task using the dataflow circuitry.

Enabling accelerated processing units to perform dataflow execution

Dynamically coalescing atomic memory operations for memory-local computing is disclosed. In an embodiment, it is determined whether a first atomic memory access and a second atomic memory access are candidates for coalescing. In response to a triggering event, the atomic memory accesses that are candidates for coalescing are coalesced in a cache prior to requesting memory-local processing by a memory-local compute unit. The atomic memory accesses may be coalesced in the same cache line or atomic memory accesses in different cache lines may be coalesced using a multicast memory-local processing command.

Growing community of inventors

Champaign, IL, United States of America

Johnathan R Alsop