Advanced Micro Devices Corporation

Alexandru Dutu

Bradford Michael Beckmann

David Allen Wood

Matthew David Sinclair

Nuwan S Jayasena

Sooraj Puthoor

Niti Madan

Yasuko Eckert

Sergey Blagodurov

Shaizeen Dilawarhusen Aga

Anthony T Gutierrez

Johnathan R Alsop

Michael Warren Boyer

Matthew R Poremba

Marcus Nathaniel Chow

Vaibhav Ramakrishnan Ramachandran

Scheduling processing-in-memory transactions is described. In accordance with the described techniques, a memory controller receives a transaction header from a host, where the transaction header describes a number of operations to be executed by a processing-in-memory component as part of performing the transaction. The memory controller adds the transaction header to a buffer and sends either an acknowledgement message or a negative acknowledgement message to the host, based on a current load of the processing-in-memory component. The acknowledgement message causes the host to send operations of the transaction for execution by the processing-in-memory component and the negative acknowledgement message causes the host to refrain from sending the operations of the transaction for execution by the processing-in-memory component.

Scheduling processing-in-memory transactions

A memory includes at least two portions of memory circuitry, a first processor in memory (PIM) functional block, and a second PIM functional block, each PIM functional block being associated with a respective different portion of the memory circuitry. In operation, the first PIM functional block copies data from a source location in a first portion of memory accessible to the first PIM functional block. The first PIM functional block then provides the data to the second PIM functional block. The second PIM functional block acquires and stores the data in a destination location in a second portion of memory accessible to the second PIM functional block. The second PIM functional block next performs one or more PIM operations using the data.

Providing data from portions of a memory to processors in memory (PIMs) in an electronic device

Scheduling requests of multiple processing-in-memory threads and requests of multiple non-processing-in-memory threads is described. In accordance with the described techniques, a memory controller receives a plurality of processing-in-memory threads and a plurality of non-processing-in-memory threads from a host. The memory controller schedules an order of execution for requests of the plurality of processing-in-memory threads and requests of the plurality of non-processing-in-memory threads based on a priority associated with each of the requests and a current operating mode of the system. Requests are maintained in queues at the memory controller and are individually assigned a priority level based on time enqueued at the memory controller. Requests of a different mode than a current operating mode of the system are delayed for scheduling until at least one different mode request is escalated to a maximum priority value, at which point the memory controller initiates a system mode switch.

Scheduling multiple processing-in-memory (PIM) threads and non-PIM threads

An approach provides indirect addressing support for PIM. Indirect PIM commands include address translation information that allows memory modules to perform indirect addressing. Processing logic in a memory module processes an indirect PIM command and retrieves, from a first memory location, a virtual address of a second memory location. The processing logic calculates a corresponding physical address for the virtual address using the address translation information included with the indirect PIM command and retrieves, from the second memory location, a virtual address of a third memory location. This process is repeated any number of times until one or more indirection stop criteria are satisfied. The indirection stop criteria stop the process when work has been completed normally or to prevent errors. Implementations include the processing logic in the memory module working in cooperation with a memory controller to perform indirect addressing.

Approach for providing indirect addressing in memory modules

An apparatus includes a memory controller that includes logic to receive a first memory request having a first request type and a second memory request having a second request type. The apparatus also includes a scheduling unit that includes logic to schedule an order of the first and second memory requests for execution based upon a first parameter value and a second parameter value. The first parameter value corresponds to a utility and energy cost for the first memory request and the second parameter value corresponds to a utility and energy cost for the second memory request.

Allocation of resources when processing at memory level through memory request scheduling

A technique for synchronizing workgroups is provided. Multiple workgroups execute a wait instruction that specifies a condition variable and a condition. A workgroup scheduler stops execution of a workgroup that executes a wait instruction and an advanced controller begins monitoring the condition variable. In response to the advanced controller detecting that the condition is met, the workgroup scheduler determines whether there is a high contention scenario, which occurs when the wait instruction is part of a mutual exclusion synchronization primitive and is detected by determining that there is a low number of updates to the condition variable prior to detecting that the condition has been met. In a high contention scenario, the workgroup scheduler wakes up one workgroup and schedules another workgroup to be woken up at a time in the future. In a non-contention scenario, more than one workgroup can be woken up at the same time.

Selective workgroup wake-up based on synchronization mechanism identification with high contention scenario

Adaptive scheduling of memory requests and processing-in-memory requests is described. In accordance with the described techniques, a memory controller receives a plurality of processing-in-memory requests and a plurality of non-processing-in-memory requests from a host. The memory controller schedules an order of execution for the plurality of processing-in-memory requests and the plurality of non-processing-in-memory requests based at least in part on a processing-in-memory request stall threshold and a non-processing-in-memory request stall threshold. In response to a system switching (e.g., from executing processing-in-memory requests to executing non-processing-in-memory requests or from executing non-processing-in-memory requests to executing processing-in-memory requests), the memory controller modifies the processing-in-memory request stall threshold and the non-processing-in-memory request stall threshold. The memory controller continues scheduling an order of execution for subsequent requests received from the host using the modified stall thresholds.

Adaptive scheduling of memory and processing-in-memory requests

A processing system monitors and synchronizes parallel execution of workgroups (WGs). One or more of the WGs perform (e.g., periodically or in response to a trigger such as an indication of oversubscription) a waiting atomic instruction. In response to a comparison between an atomic value produced as a result of the waiting atomic instruction and an expected value, WGs that fail to produce a correct atomic value are identified as being in a waiting state (e.g., waiting for a synchronization variable). Execution of WGs in the waiting state is prevented (e.g., by a context switch) until corresponding synchronization variables are released.

Workgroup synchronization and processing

Techniques for executing workgroups are provided. The techniques include executing, for a first workgroup of a first kernel dispatch, a workgroup dependency instruction that includes an indication to prioritize execution of a second workgroup of a second kernel dispatch, and in response to the workgroup dependency instruction, dispatching the second workgroup of the second kernel dispatch prior to dispatching a third workgroup of the second kernel dispatch, wherein no workgroup dependency instruction including an indication to prioritize execution of the third workgroup has been executed.

Fine-grained conditional dispatching

Dynamically coalescing atomic memory operations for memory-local computing is disclosed. In an embodiment, it is determined whether a first atomic memory access and a second atomic memory access are candidates for coalescing. In response to a triggering event, the atomic memory accesses that are candidates for coalescing are coalesced in a cache prior to requesting memory-local processing by a memory-local compute unit. The atomic memory accesses may be coalesced in the same cache line or atomic memory accesses in different cache lines may be coalesced using a multicast memory-local processing command.

Dynamically coalescing atomic memory operations for memory-local computing

A first workgroup is preempted in response to threads in the first workgroup executing a first wait instruction including a first value of a signal and a first hint indicating a type of modification for the signal. The first workgroup is scheduled for execution on a processor core based on a first context after preemption in response to the signal having the first value. A second workgroup is scheduled for execution on the processor core based on a second context in response to preempting the first workgroup and in response to the signal having a second value. A third context it is prefetched into registers of the processor core based on the first hint and the second value. The first context is stored in a first portion of the registers and the second context is prefetched into a second portion of the registers prior to preempting the first workgroup.

Cooperative workgroup scheduling and context prefetching based on predicted modification of signal values

A technique for synchronizing workgroups is provided. The techniques comprise detecting that one or more non-executing workgroups are ready to execute, placing the one or more non-executing workgroups into one or more ready queues based on the synchronization status of the one or more workgroups, detecting that computing resources are available for execution of one or more ready workgroups, and scheduling for execution one or more ready workgroups from the one or more ready queues in an order that is based on the relative priority of the ready queues.

Enhanced atomics for workgroup synchronization

A technique for implementing synchronization monitors on an accelerated processing device ('APD') is provided. Work on an APD includes workgroups that include one or more wavefronts. All wavefronts of a workgroup execute on a single compute unit. A monitor is a synchronization construct that allows workgroups to stall until a particular condition is met. Responsive to all wavefronts of a workgroup executing a wait instruction, the monitor coordinator records the workgroup in an 'entry queue.' The workgroup begins saving its state to a general APD memory and, when such saving is complete, the monitor coordinator moves the workgroup to a &#x201c;condition queue.&#x201d; When the condition specified by the wait instruction is met, the monitor coordinator moves the workgroup to a &#x201c;ready queue,&#x201d; and, when sufficient resources are available on a compute unit, the APD schedules the ready workgroup for execution on a compute unit.

Growing community of inventors

Bellevue, WA, United States of America

Alexandru Dutu