Advanced Micro Devices Corporation

Ati Technologies Ulc

Globalfoundries Inc.

Benjamin T Sander

James K Pickett

Michael A Filippo

Kevin Michael Lepak

Keith Lowery

Michael C Houston

Newton Cheung

Anthony Asaro

Mitchell K Alsup

Gongxian Jeffrey Cheng

Michael Mantor

Mark Fowler

Brian D McMinn

Philip J Rogers

Gregory William Smaus

Teik-Chung Tan

Vydhyanathan Kalyanasundharam

William Alexander Hughes

Michael T Clark

David Edward Kroesche

Kiran Bondalapati

Philip Enrique Madrid

Paul Blinzer

Roger Dwain Isaac

Raja Koduri

Krishnan V Ramani

Swapnil Pradipkumar Sakharshete

Ramsey W Haddad

Arun Radhakrishnan

Rama S Gopal

Sridhar P Subramanian

Faisal A Syed

Ali Ibrahim

Michael E Tuuk

Leonel Lozano

Bharath Narasimha Swamy

Swamy Punyamurtula

Greg Smaus

Gregory P Rodgers

Adam Duley

Teik-Chang Tan

Ashish Panday

Shreyas Ramalingam

Christopher Svec

Michael Filippo

Systems, apparatuses, and methods for achieving higher cache hit rates for machine learning models are disclosed. When a processor executes a given layer of a machine learning model, the processor generates and stores activation data in a cache subsystem a forward or reverse manner. Typically, the entirety of the activation data does not fit in the cache subsystem. The processor records the order in which activation data is generated for the given layer. Next, when the processor initiates execution of a subsequent layer of the machine learning model, the processor processes the previous layer's activation data in a reverse order from how the activation data was generated. In this way, the processor alternates how the layers of the machine learning model process data by either starting from the front end or starting from the back end of the array.

Methods for increasing cache hit rates for neural networks

A processor maintains an access log indicating a stream of cache misses at a cache of the processor. In response to each of at least a subset of cache misses at the cache, the processor records a corresponding entry in the access log, indicating a physical memory address of the memory access request that resulted in the corresponding miss. In addition, the processor maintains an address translation log that indicates a mapping of physical memory addresses to virtual memory addresses. In response to an address translation (e.g., a page walk) that translates a virtual address to a physical address, the processor stores a mapping of the physical address to the corresponding virtual address at an entry of the address translation log. Software executing at the processor can use the two logs for memory management.

Access log and address translation log for a processor

A processor uses the same virtual address space for heterogeneous processing units of the processor. The processor employs different sets of page tables for different types of processing units, such as a CPU and a GPU, wherein a memory management unit uses each set of page tables to translate virtual addresses of the virtual address space to corresponding physical addresses of memory modules associated with the processor. As data is migrated between memory modules, the physical addresses in the page tables can be updated to reflect the physical location of the data for each processing unit.

Shared virtual address space for heterogeneous processors

A processing system includes a first socket, a second socket, and an interface between the first socket and the second socket. A first memory is associated with the first socket and a second memory is associated with the second socket. The processing system also includes a controller for the first memory. The controller is to receive a first request for a first memory transaction with the second memory and perform the first memory transaction along a path that includes the interface and bypasses at least one second cache associated with the second memory.

Caching policies for processing units on multiple sockets

A memory manager of a processor identifies a block of data for eviction from a first memory module to a second memory module. In response, the processor copies only those portions of the data block that have been identified as modified portions to the second memory module. The amount of data to be copied is thereby reduced, improving memory management efficiency and reducing processor power consumption.

Selective data copying between memory modules

Embodiments described herein include a system, a computer-readable medium and a computer-implemented method for processing a system call (SYSCALL) request. The SYSCALL request from an invisible processing device is stored in a queueing mechanism that is accessible to a visible processing device, where the visible processing device is visible to an operating system and the invisible processing device is invisible to the operating system. The SYSCALL request is processed using the visible processing device, and the invisible processing device is notified using a notification mechanism that the SYSCALL request was processed.

System call queue between visible and invisible computing devices

Methods, systems and computer-readable mediums for task scheduling on an accelerated processing device (APD) are provided. In an embodiment, a method comprises: enqueuing one or more tasks in a memory storage module based on the APD; using a software-based enqueuing module; and dequeuing the one or more tasks from the memory storage module using a hardware-based command processor, wherein the command processor forwards the one or more tasks to the shader cote.

Heterogeneous enqueuing and dequeuing mechanism for task scheduling

A processor device includes a cache and a memory storing a set of counters. Each counter of the set is associated with a corresponding block of a plurality of blocks of the cache. The processor device further includes a cache access monitor to, for each time quantum for a series of one or more time quanta, increment counter values of the set of counters based on accesses to the corresponding blocks of the cache. The processor device further includes a transfer engine to, after completion of each time quantum, transfer the counter values of the set of counters for the time quantum to a corresponding location in a system memory.

Cache access statistics accumulation for cache line replacement selection

A method, system and article of manufacture for balancing a workload on heterogeneous processing devices. The method comprising accessing a memory storage of a processor of one type by a dequeuing entity associated with a processor of a different type, identifying a task from a plurality of tasks within the memory that can be processed by the processor of the different type, synchronizing a plurality of dequeuing entities capable of accessing the memory storage, and dequeuing the task form the memory storage.

Dynamic work partitioning on heterogeneous processing devices

A programming model for a processor accelerator allows accelerated functions to be called from a main program directly without a management API for the accelerator. A compiler automatically generates wrapper source code for each accelerator function called by the application source code. The wrapper code is compiled, together with the accelerator source code, to generate an object file that is linked to an object file for the main program. By automatically generating the wrapper code, a programmer can simply and directly invoke accelerator functions without the use of a complex management API. In addition, because the wrapper code for the accelerator is generated automatically, a standard compiler can be used to compile the main program, using standard linkage conventions.

Automatic source code generation for accelerated function calls

Methods, systems and computer-readable mediums for task scheduling on an accelerated processing device (APD) are provided. In an embodiment, a method comprises: enqueuing one or more tasks in a memory storage module based on the APD; using a software-based enqueuing module; and dequeuing the one or more tasks from the memory storage module using a hardware-based command processor, wherein the command processor forwards the one or more tasks to the shader core.

Provided herein is a method for optimizing communication for system calls. The method includes storing a system call for each work item in a wavefront and transmitting said stored system calls to a processor for execution. The method also includes receiving a result to each work item in the wavefront responsive to said transmitting.

Optimizing communication of system call requests

A system, method and article of manufacture for an accelerated processing device (APD) to request a central processing unit (CPU) to process a task, comprising enqueuing a plurality of tasks on a queue using the APD, generating a user-level interrupt and transmitting to the CPU the plurality of tasks in the queue using an interrupt handler associated with a CPU thread.

Computer system interrupt handling

A system and method for efficient data prefetching. A data stream stored in lower-level memory comprises a contiguous block of data used in a computer program. A prefetch unit in a processor detects a data stream by identifying a sequence of storage accesses referencing a contiguous blocks of data in a monotonically increasing or decreasing manner. After a predetermined training period for a given data stream, the prefetch unit prefetches a portion of the given data stream from memory without write permission, in response to an access that does not request write permission. Also, after the training period, the prefetch unit prefetches a portion of the given data stream from lower-level memory with write permission, in response to determining there has been a prior access to the given data stream that requests write permission subsequent to a number of cache misses reaching a predetermined threshold.

Store aware prefetching for a datastream

Growing community of inventors

Austin, TX, United States of America

Benjamin T Sander