Advanced Micro Devices Corporation

Micron Technology Incorporated

Hewlett-Packard Development Company, L.p.

Arm Limited

University of Michigan

Ati Technologies Ulc

David Andrew Roberts

Robert Michael Walker

Joseph Thomas Pawlowski

Michael Ignatowski

Nuwan S Jayasena

J Thomas Pawlowski

Steven E Raasch

Mitesh Ramesh Meswani

Elliot H Mednick

Amin Farmahini-Farahani

Parthasarathy Ranganathan

Greg Sadowski

Andrew Gregory Kegel

Sergey Blagodurov

Jichuan Chang

Elliott Clifford Cooper-Balis

Aliasger Tayeb Zaidy

Amin Farmahini Farahani

Gabriel H Loh

Vilas K Sridharan

Kevin Y Cheng

Tony M Brewer

Manish Satyapal Gupta

Mark Richard Nutter

Trevor N Mudge

Ganesh Suryanarayan Dasika

Steven Moyer

Arkaprava Basu

Nabeel Meeramohideen Mohamed

Dean E Gonzales

Patrick Michael Sheridan

John Robert Slice

Haojie Ye

Shenghsun Cho

Prashant Jayaprakash Nair

Lukasz Burzawa

Sean Stephen Eilert

Anthony Asaro

Mauricio Breternitz, Jr

Mehul A Shah

Aaron John Nygren

Sudhanva Gurumurthi

David Boles

Patrick Alan Estep

Mark Henry Oskin

Ehsan Fatehi

Daniel I Lowell

William C Brantley

David John Cownie

Meenakshi Sundaram Bhaskaran

Nathan Hu

John Sontag

Described apparatuses and methods track access metadata pertaining to activity within respective address ranges. The access metadata can be used to inform prefetch operations within the respective address ranges. The prefetch operations may involve deriving access patterns from access metadata covering the respective ranges. Suitable address range sizes for accurate pattern detection, however, can vary significantly from region to region of the address space based on, inter alia, workloads produced by programs utilizing the regions. Advantageously, the described apparatuses and methods can adapt the address ranges covered by the access metadata for improved prefetch performance. A data structure may be used to manage the address ranges in which access metadata are tracked. The address ranges can be adapted to improve prefetch performance through low-overhead operations implemented within the data structure. The data structure can encode hierarchical relationships that ensure the resulting address ranges are distinct.

Adaptive address tracking

Methods, systems, and devices for access heatmap implementations at a host device are described. A host device may leverage access operation monitoring that is performed at a memory device, including various examples of signaling and management of monitoring configurations. For example, a memory device may maintain a storage location for tracking access operation occurrence, for which access operations of a given address may be mapped to multiple fields, and for which each field may be associated with access operations of a respective subset of the addresses. In some examples, such registers may be configured or accessed based on indications (e.g., commands, requests) from a host device, which may support dynamic access operation monitoring that is responsive to various operating conditions. In some examples, the host device may perform evaluations based on such minimum values associated with respective addresses to determine a distribution of data across various portions of memory.

Access heatmap implementations at a host device

Disclosed in some examples are improved address prediction and memory preloading that leverages next-delta prediction and/or far-delta prediction for scheduling using a DNN. Previous memory access sequence data that identify one or more memory addresses previously accessed by one or more processors of a system may be processed and then converted into a sequence of delta values. The sequence of delta values are then mapped to one or more classes that are then input to a DNN. The DNN then outputs a predicted future class identifier sequence that represents addresses that the DNN predicts will be accessed by the processor in the future. The predicted future class identifier sequence is then converted back to a predicted delta value sequence and back into a set of one or more predicted addresses.

Delta predictions for page scheduling

A cache memory can maintain multiple cache lines and each cache line can include a data field, an encryption status attribute, and an encryption key attribute. The encryption status attribute can indicate whether the data field in the corresponding cache line includes encrypted or unencrypted data and the encryption key attribute can include an encryption key identifier for the corresponding cache line. In an example, a cryptographic controller can access keys from a key table to selectively encrypt or unencrypt cache data. Infrequently accessed cache data can be maintained as encrypted data, and more frequently accessed cache data can be maintained as unencrypted data. In some examples, different cache lines in the same cache memory can be maintained as encrypted or unencrypted data, and different cache lines can use respective different encryption keys.

Selective cache line memory encryption

Systems, apparatuses, and methods for memory management are described. For example, these may include a first memory level including memory pages in a memory array, a second memory level including a cache, a pre-fetch buffer, or both, and a memory controller that determines state information associated with a memory page in the memory array targeted by a memory access request. The state information may include a first parameter indicative of a current activation state of the memory page and a second parameter indicative of statistical likelihood (e.g., confidence) that a subsequent memory access request will target the memory page. The memory controller may disable storage of data associated with the memory page in the second memory level when the first parameter associated with the memory page indicates that the memory page is activated and the second parameter associated with the memory page is greater than or equal to a threshold.

Memory-aware pre-fetching and cache bypassing systems and methods

Disclosed in some examples, are methods, systems, and machine-readable mediums in which application state is saved using in-memory versioning in a shared memory pool of disaggregated memory. By utilizing a disaggregated memory pool, the processing resources may be on separate devices than the memory those resources are using. As a result of this architecture, a failure of hardware of processing resources or an application does not necessarily also cause the hardware resources of the memory devices to fail. This allows a standby application executing on standby processing resources to quickly resume execution when a primary application fails by utilizing the memory pool assigned to the primary application in the memory pool.

Fast failure recovery of applications

Devices and techniques for continuous in-memory versioning are described herein. A memory subsystem includes a memory device configured to store a first data unit, a second data unit, and a third data unit, wherein the first, second, and third data units have a set of physical memory locations on the memory device, and metadata associated with the first, second, and third data units, the metadata including state information and a dirty commit timestamp; and a processing device, operatively coupled to the memory device, the processing device configured to: receive, from a host system, a first memory command associated with a logical memory address, the logical memory address mapped to the set of physical memory locations of the memory device; and in response to receiving the first memory command, perform a data operation on the first, second, or third data unit based on the state information and the dirty commit timestamp.

Systems and methods for continuous in-memory versioning

Disclosed in some examples are systems, devices, machine-readable mediums, and methods for customizing an in-memory versioning mode for each memory location according to a predicted access behavior to optimize memory device performance. Usage data in a previous time period may be utilized along with policy rules to determine whether to configure a particular memory address as a zero copy or direct copy mode. For example, memory addresses that are read frequently may be configured as direct copy mode to reduce the read latency penalty. This improves the functioning of the memory system by reducing read latency for memory addresses that are frequently read but written to less frequently, and reduces write latency for memory locations that are frequently written to, but not read as frequently.

Adaptive control for in-memory versioning

Methods, systems, and devices for performance monitoring for a memory system are described. A memory system may use a set of counters to determine state information for the memory system. The memory system may also use a set of timers to determine latency information for the memory system. In response to a request for performance information, the memory system may transmit state information, latency information, or both to a host system.

Performance monitoring for a memory system

A system includes a memory device and a processing device coupled to the memory device, and the processing device is to perform operations including determining, by monitoring accesses to the memory device, a plurality of values of one or more memory usage statistics reflecting memory usage by a plurality of requestors connected to the memory sub-system; generating memory usage data by processing the plurality of values of the one or more memory usage statistics; and transmitting, to a requestor of the plurality of requestors, the memory usage data.

Memory phase monitoring and scheduling system using memory usage statistics

Methods, systems, and devices for access heatmap generation at a memory device are described. In some examples, a memory device may maintain a register for tracking access operation occurrence, for which access operations of an address of the memory device may be mapped to multiple fields of the register. In some cases, in response to a first access operation performed on a first address of the memory device, the memory device may increment a first field and a second field of the register and, in response to a second access operation performed on a second address of the memory device, the memory device may increment the first field and a third field of the register. In some examples, the memory device may maintain a second register having a set of fields that each indicate a respective address for which an access occurrence satisfies a threshold.

Techniques to generate access heatmaps at a memory device

Systems, apparatuses, and methods related to memory access statistics monitoring are described. A host is configured to map pages of memory for applications to a number of memory devices coupled thereto. A first memory device comprises a monitoring component configured to monitor access statistics of pages of memory mapped to the first memory device. A second memory device does not include a monitoring component capable of monitoring access statistics of pages of memory mapped thereto. The host is configured to map a portion of pages of memory for an application to the first memory device in order to obtain access statistics corresponding to the portion of pages of memory upon execution of the application despite there being space available on the second memory device and adjust mappings of the pages of memory for the application based on the obtained access statistics corresponding to the portion of pages.

Memory access statistics monitoring

A method and apparatus physically partitions clean and dirty cache lines into separate memory partitions, such as one or more banks, so that during low power operation, a cache memory controller reduces power consumption of the cache memory containing the clean only data. The cache memory controller controls refresh operation so that data refresh does not occur for clean data only banks or the refresh rate is reduced for clean data only banks. Partitions that store dirty data can also store clean data, however other partitions are designated for storing only clean data so that the partitions can have their refresh rate reduced or refresh stopped for periods of time. When multiple DRAM dies or packages are employed, the partition can occur on a die or package level as opposed to a bank level within a die.

Method and apparatus for controlling cache line storage in cache memory

Systems, devices, and methods related to a Deep Learning Accelerator and memory are described. For example, the accelerator can have processing units to perform at least matrix computations of an artificial neural network via execution of instructions. The processing units have a local memory store operands of the instructions. The accelerator can access a random access memory via a system buffer, or without going through the system buffer. A fetch instruction can request an item, available at a memory address in the random access memory, to be loaded into the local memory at a local address. The fetch instruction can include a hint for the caching of the item in the system buffer. During execution of the instruction, the hint can be used to determine whether to load the item through the system buffer or to bypass the system buffer in loading the item.

Caching techniques for deep learning accelerator

Growing community of inventors

Boxborough, MA, United States of America

David Andrew Roberts