Intel Corporation

Varghese George

Subramaniam M Maiyuran

Altug Koker

Sanjeev S Jahagirdar

Joydeep Ray

Elmoustapha Ould-Ahmed-Vall

Abhishek R Appu

Aravindh V Anantaraman

Mike MacPherson

Valentin Andrei

Lakshminarayanan Striramassarma

Vasanth Ranganathan

Prasoonkumar Surti

Sungye Kim

Nicolas C Galoppo Von Borries

Vikranth Vemulapalli

Ashutosh Garg

Scott R Janus

Jorge E Parra

Chandra S Gurram

Jose P Allarey

Ben J Ashbaugh

Inder M Sodhi

Stephen Anthony Fischer

Supratim Pal

Shubra Marwaha

Guei-Yuan Lueh

Alon Naveh

Josh B Mastronarde

William B Sadler

Pattabhiraman K

Iqbal Rajwani

Naveen Matam

Selvakumar Panneer

Brent E Insko

Arthur Hunter, Jr

Shai Rotem

Robert Milstrey

Oren Lamdan

Murali Ramadoss

Jonathan D Pearce

Darin Starkey

Stephen H Gunther

Dipankar Das

Jill MacDonald Boyce

Kamal Sinha

Ankur N Shah

Adam T Lake

Carl S Marshall

Deepak Shashidhar Vembar

Karthik Veeramani

Niranjan L Cooray

Naveen K Mellempudi

Fangwen Fu

Tomer Ziv

Yoav Harel

Rajesh M Sankaran

David M Puffer

Vladimir M Pentkovski

Wei-Yu Chen

William Rash

Vallabhajosyula S Somayazulu

Hugues Labbe

Deborah T Marr

Jeffrey R Wilcox

Stephen Junkins

Xinmin Tian

Nilesh K Jain

Deep K Buch

Sherine Abdelhak

Zhijun Lei

Chunhui Mei

Glen J Anderson

Bret L Toll

James A Valerio

Robert L Farrell

Pratik M Marolia

Leslie E Cline

Yan Li

Soethiha Soe

Christopher J Hughes

Efraim Rotem

David Wyatt

Hong Jiang

Alexander F Heinecke

Ravishankar Iyer

Eric C Samson

Brian K Langendorf

Dan Baum

Mike B Macpherson

Robert S Chappell

Michael T Klinglesmith

Achmed Rumi Zahir

Ronen Zohar

Xia Dai

Michael J Espig

Michael Zelikson

Steven Spangler

Douglas Raymond Moran

An apparatus to facilitate utilizing structured sparsity in systolic arrays is disclosed. The apparatus includes a processor comprising a systolic array to receive data from a plurality of source registers, the data comprising unpacked source data, structured source data that is packed based on sparsity, and metadata corresponding to the structured source data; identify portions of the unpacked source data to multiply with the structured source data, the portions of the unpacked source data identified based on the metadata; and output, to a destination register, a result of multiplication of the portions of the unpacked source data and the structured source data.

Utilizing structured sparsity in systolic arrays

A processing apparatus is described herein that includes a general-purpose parallel processing engine comprising a matrix accelerator including one or more systolic arrays, at least one of the one or more systolic arrays comprising multiple pipeline stages, each pipeline stage of the multiple pipeline stages including multiple processing elements, the multiple processing elements configured to perform processing operations on input matrix elements based on output sparsity metadata. The output sparsity metadata indicates to the multiple processing elements to bypass multiplication for a first row of elements of a second matrix and multiply a second row of elements of the second matrix with a column of matrix elements of a first matrix.

Systolic array having support for output sparsity

Embodiments described herein provide techniques to enable the dynamic reconfiguration of memory on a general-purpose graphics processing unit. One embodiment described herein enables dynamic reconfiguration of cache memory bank assignments based on hardware statistics. One embodiment enables for virtual memory address translation using mixed four kilobyte and sixty-four kilobyte pages within the same page table hierarchy and under the same page directory. One embodiment provides for a graphics processor and associated heterogenous processing system having near and far regions of the same level of a cache hierarchy.

Dynamic memory reconfiguration

Embodiments are generally directed to memory prefetching in multiple GPU environment. An embodiment of an apparatus includes multiple processors including a host processor and multiple graphics processing units (GPUs) to process data, each of the GPUs including a prefetcher and a cache; and a memory for storage of data, the memory including a plurality of memory elements, wherein the prefetcher of each of the GPUs is to prefetch data from the memory to the cache of the GPU; and wherein the prefetcher of a GPU is prohibited from prefetching from a page that is not owned by the GPU or by the host processor.

Prefetch status notification for memory prefetching

A processing apparatus includes a general-purpose parallel processing engine including a set of multiple processing elements including a single precision floating-point unit, a double precision floating point unit, and an integer unit; a matrix accelerator including one or more systolic arrays; a first register file coupled with a first read control circuit, wherein the first read control circuit couples with the set of multiple processing elements and the matrix accelerator to arbitrate read requests to the first register file from the set of multiple processing elements and the matrix accelerator; and a second register file coupled with a second read control circuit, wherein the second read control circuit couples with the matrix accelerator to arbitrate read requests to the second register file from the matrix accelerator and limit access to the second register file by the set of multiple processing elements.

Register file for systolic array

Embodiments described herein provide techniques to facilitate instruction-based control of memory attributes. One embodiment provides a graphics processor comprising a processing resource, a memory device, a cache coupled with the processing resources and the memory, and circuitry to process a memory access message received from the processing resource. The memory access message enables access to data of the memory device. To process the memory access message, the circuitry is configured to determine one or more cache attributes that indicate whether the data should be read from or stored the cache. The cache attributes may be provided by the memory access message or stored in state data associated with the data to be accessed by the access message.

Implicit fence for write messages

An apparatus to facilitate efficient data sharing for graphics data processing operations is disclosed. The apparatus includes a processing resource to generate a stream of instructions, an L1 cache communicably coupled to the processing resource and comprising an on-page detector circuit to determine that a set of memory requests in the stream of instructions access a same memory page; and set a marker in a first request of the set of memory requests; and arbitration circuitry communicably coupled to the L1 cache, the arbitration circuitry to route the set of memory requests to memory comprising the memory page and to, in response to receiving the first request with the marker set, remain with the processing resource to process the set of memory requests.

Efficient data sharing for graphics data processing operations

Embodiments described herein include, software, firmware, and hardware logic that provides techniques to perform arithmetic on sparse data via a systolic processing unit. Embodiment described herein provided techniques to detect zero value elements within a vector or a set of packed data elements output by a processing resource and generate metadata to indicate a location of the zero value elements within the plurality of data elements.

Sparse optimizations for a matrix accelerator architecture

Apparatuses including general-purpose graphics processing units having on chip dense memory for temporal buffering are disclosed. In one embodiment, a graphics multiprocessor includes a plurality of compute engines to perform first computations to generate a first set of data, cache for storing data, and a high density memory that is integrated on chip with the plurality of compute engines and the cache. The high density memory to receive the first set of data, to temporarily store the first set of data, and to provide the first set of data to the cache during a first time period that is prior to a second time period when the plurality of compute engines will use the first set of data for second computations.

On chip dense memory for temporal buffering

Methods and apparatus relating to data initialization techniques. In an example, an apparatus comprises a processor to read one or more metadata codes which map to one or more cache lines in a cache memory and invoke a random number generator to generate random numerical data for the one or more cache lines in response to a determination that the one more metadata codes indicate that the cache lines are to contain random numerical data. Other embodiments are also disclosed and claimed.

Data initialization techniques

An apparatus to facilitate supporting 8-bit floating point format operands in a computing architecture is disclosed. The apparatus includes a processor comprising: a decoder to decode an instruction fetched for execution into a decoded instruction, wherein the decoded instruction is a matrix instruction that operates on 8-bit floating point operands to cause the processor to perform a parallel dot product operation; a controller to schedule the decoded instruction and provide input data for the 8-bit floating point operands in accordance with an 8-bit floating data format indicated by the decoded instruction; and systolic dot product circuitry to execute the decoded instruction using systolic layers, each systolic layer comprises one or more sets of interconnected multipliers, shifters, and adder, each set of multipliers, shifters, and adders to generate a dot product of the 8-bit floating point operands.

Supporting 8-bit floating point format operands in a computing architecture

One embodiment provides a graphics processor comprising a block of execution resources, a cache memory, a cache memory prefetcher, and circuitry including a programmable neural network unit, the programmable neural network unit comprising a network hardware block including circuitry to perform neural network operations and activation operations for a layer of a neural network, the programmable neural network unit addressable by cores within the block of graphics cores and the neural network hardware block configured to perform operations associated with a neural network configured to determine a prefetch pattern for the cache memory prefetcher.

Graphics architecture including a neural network pipeline

Graphics processing systems and methods are described. A graphics processing apparatus may comprise one or more graphics processing engines, a memory, a memory management unit (MMU) including a GPU second level page table and GPU dirty bit tracking, and a provisioning agent to receive a request from a virtual machine monitor (VMM) to provision a subcluster of graphics processing apparatuses, the subcluster including a plurality of graphics processing engines from a plurality of graphics processing apparatuses connected using a scale-up fabric, provision the scale-up fabric to route data within the subcluster of graphics processing apparatuses, and provision a plurality of resources on the graphics processing apparatus for the subcluster based on the request from the VMM.

Virtualization and multi-tenancy support in graphics processors

Apparatuses to synchronize lanes that diverge or threads that drift are disclosed. In one embodiment, a graphics multiprocessor includes a queue having an initial state of groups with a first group having threads of first and second instruction types and a second group having threads of the first and second instruction types. A regroup engine (or regroup circuitry) regroups threads into a third group having threads of the first instruction type and a fourth group having threads of the second instruction type.

Systems and methods for synchronization of multi-thread lanes

Systems and methods for improving cache efficiency and utilization are disclosed. In one embodiment, a graphics processor includes processing resources to perform graphics operations and a cache controller of a cache coupled to the processing resources. The cache controller is configured to control cache priority by determining whether default settings or an instruction will control cache operations for the cache.

Systems and methods for improving cache efficiency and utilization

Growing community of inventors

Folsom, CA, United States of America

Varghese George