Intel Corporation

Mike MacPherson

Varghese George

Abhishek R Appu

Joydeep Ray

Altug Koker

Subramaniam M Maiyuran

Elmoustapha Ould-Ahmed-Vall

Aravindh V Anantaraman

Valentin Andrei

Vasanth Ranganathan

Nicolas C Galoppo Von Borries

Prasoonkumar Surti

Lakshminarayanan Striramassarma

Scott R Janus

Sungye Kim

Ben J Ashbaugh

Vikranth Vemulapalli

Arthur Hunter, Jr

William B Sadler

Murali Ramadoss

Jonathan D Pearce

Kamal Sinha

Brent E Insko

Ashutosh Garg

Ankur N Shah

Selvakumar Panneer

Yoav Harel

David M Puffer

Pattabhiraman K

Fangwen Fu

Sherine Abdelhak

Saurabh Tangri

Jill MacDonald Boyce

Feng Chen

Adam T Lake

Carl S Marshall

Xiaoming Chen

Rajkishore Barik

Vallabhajosyula S Somayazulu

Deepak Shashidhar Vembar

Karthik Veeramani

Hugues Labbe

Stephen Junkins

Jorge E Parra

Chandra S Gurram

Zhijun Lei

Zhengmin Li

Jayakrishna P S

Darrel K Palke

Guei-Yuan Lueh

John Charles Weast

Linda L Hurd

Dukhwan Kim

James A Valerio

Niranjan L Cooray

Brian T Lewis

Murali Sundaresan

Jeffrey R Jackson

Supratim Pal

Jiasheng Chen

Marian Alin Petre

Shailesh Shah

Balaji Vembu

Sanjeev S Jahagirdar

Narayan Srinivasa

Anbang Yao

Ravishankar Iyer

Eriko Nurvitadhi

Eric C Samson

Tatiana Shpeisman

Narayan Biswal

Ping Tak Tang

Nadathur Rajagopalan Satish

Stanley J Baran

Tsung-Han Lin

Dipankar Das

Josh B Mastronarde

Michael S Strickland

Farshad Akhbari

Gokcen Cilingir

Nilesh V Shah

Wei-Yu Chen

Naveen K Mellempudi

Barnan Das

Archie Sharma

Mayuresh M Varerkar

Nilesh K Jain

Shubra Marwaha

Timothy R Bauer

Sasikanth Avancha

Shubh Shah

Konrad Trifunovic

Neelay Pandit

Sudhakar Kamma

Weiyu Chen

Junjie Gu

Hong Bin Liao

Praneetha Kotha

Nilay Mistry

A processing apparatus is described herein that includes a general-purpose parallel processing engine comprising a matrix accelerator including one or more systolic arrays, at least one of the one or more systolic arrays comprising multiple pipeline stages, each pipeline stage of the multiple pipeline stages including multiple processing elements, the multiple processing elements configured to perform processing operations on input matrix elements based on output sparsity metadata. The output sparsity metadata indicates to the multiple processing elements to bypass multiplication for a first row of elements of a second matrix and multiply a second row of elements of the second matrix with a column of matrix elements of a first matrix.

Systolic array having support for output sparsity

Embodiments are generally directed to memory prefetching in multiple GPU environment. An embodiment of an apparatus includes multiple processors including a host processor and multiple graphics processing units (GPUs) to process data, each of the GPUs including a prefetcher and a cache; and a memory for storage of data, the memory including a plurality of memory elements, wherein the prefetcher of each of the GPUs is to prefetch data from the memory to the cache of the GPU; and wherein the prefetcher of a GPU is prohibited from prefetching from a page that is not owned by the GPU or by the host processor.

Prefetch status notification for memory prefetching

A processing apparatus includes a general-purpose parallel processing engine including a set of multiple processing elements including a single precision floating-point unit, a double precision floating point unit, and an integer unit; a matrix accelerator including one or more systolic arrays; a first register file coupled with a first read control circuit, wherein the first read control circuit couples with the set of multiple processing elements and the matrix accelerator to arbitrate read requests to the first register file from the set of multiple processing elements and the matrix accelerator; and a second register file coupled with a second read control circuit, wherein the second read control circuit couples with the matrix accelerator to arbitrate read requests to the second register file from the matrix accelerator and limit access to the second register file by the set of multiple processing elements.

Register file for systolic array

Embodiments described herein provide techniques to facilitate instruction-based control of memory attributes. One embodiment provides a graphics processor comprising a processing resource, a memory device, a cache coupled with the processing resources and the memory, and circuitry to process a memory access message received from the processing resource. The memory access message enables access to data of the memory device. To process the memory access message, the circuitry is configured to determine one or more cache attributes that indicate whether the data should be read from or stored the cache. The cache attributes may be provided by the memory access message or stored in state data associated with the data to be accessed by the access message.

Implicit fence for write messages

Embodiments described herein include, software, firmware, and hardware logic that provides techniques to perform arithmetic on sparse data via a systolic processing unit. Embodiment described herein provided techniques to detect zero value elements within a vector or a set of packed data elements output by a processing resource and generate metadata to indicate a location of the zero value elements within the plurality of data elements.

Sparse optimizations for a matrix accelerator architecture

Apparatuses including general-purpose graphics processing units having on chip dense memory for temporal buffering are disclosed. In one embodiment, a graphics multiprocessor includes a plurality of compute engines to perform first computations to generate a first set of data, cache for storing data, and a high density memory that is integrated on chip with the plurality of compute engines and the cache. The high density memory to receive the first set of data, to temporarily store the first set of data, and to provide the first set of data to the cache during a first time period that is prior to a second time period when the plurality of compute engines will use the first set of data for second computations.

On chip dense memory for temporal buffering

Methods and apparatus relating to data initialization techniques. In an example, an apparatus comprises a processor to read one or more metadata codes which map to one or more cache lines in a cache memory and invoke a random number generator to generate random numerical data for the one or more cache lines in response to a determination that the one more metadata codes indicate that the cache lines are to contain random numerical data. Other embodiments are also disclosed and claimed.

Data initialization techniques

One embodiment provides a graphics processor comprising a block of execution resources, a cache memory, a cache memory prefetcher, and circuitry including a programmable neural network unit, the programmable neural network unit comprising a network hardware block including circuitry to perform neural network operations and activation operations for a layer of a neural network, the programmable neural network unit addressable by cores within the block of graphics cores and the neural network hardware block configured to perform operations associated with a neural network configured to determine a prefetch pattern for the cache memory prefetcher.

Graphics architecture including a neural network pipeline

Systems and methods for improving cache efficiency and utilization are disclosed. In one embodiment, a graphics processor includes processing resources to perform graphics operations and a cache controller of a cache coupled to the processing resources. The cache controller is configured to control cache priority by determining whether default settings or an instruction will control cache operations for the cache.

Systems and methods for improving cache efficiency and utilization

Embodiments are generally directed to graphics processor data access and sharing. An embodiment of an apparatus includes a circuit element to produce a result in processing of an application; a load-store unit to receive the result and generate pre-fetch information for a cache utilizing the result; and a prefetch generator to produce prefetch addresses based at least in part on the pre-fetch information; wherein the load-store unit is to receive software assistance for prefetching, and wherein generation of the pre-fetch information is based at least in part on the software assistance.

Graphics processor data access and sharing

Embodiments described herein include software, firmware, and hardware logic that provides techniques to perform arithmetic on sparse data via a systolic processing unit. One embodiment provides for data aware sparsity via compressed bitstreams. One embodiment provides for block sparse dot product instructions. One embodiment provides for a depth-wise adapter for a systolic array.

Architecture for block sparse operations on a systolic array

Methods and apparatus relating to techniques for multi-tile memory management. In an example, a graphics processor includes an interposer, a first chiplet coupled with the interposer, the first chiplet including a graphics processing resource and an interconnect network coupled with the graphics processing resource, cache circuitry coupled with the graphics processing resource via the interconnect network, and a second chiplet coupled with the first chiplet via the interposer, the second chiplet including a memory-side cache and a memory controller coupled with the memory-side cache. The memory controller is configured to enable access to a high-bandwidth memory (HBM) device, the memory-side cache is configured to cache data associated with a memory access performed via the memory controller, and the cache circuitry is logically positioned between the graphics processing resource and a chiplet interface.

Multi-tile memory management

Embodiments are generally directed to cache structure and utilization. An embodiment of an apparatus includes one or more processors including a graphics processor; a memory for storage of data for processing by the one or more processors; and a cache to cache data from the memory; wherein the apparatus is to provide for dynamic overfetching of cache lines for the cache, including receiving a read request and accessing the cache for the requested data, and upon a miss in the cache, overfetching data from memory or a higher level cache in addition to fetching the requested data, wherein the overfetching of data is based at least in part on a current overfetch boundary, and provides for data is to be prefetched extending to the current overfetch boundary.

Cache structure and utilization

Embodiments described herein include software, firmware, and hardware logic that provides techniques to perform arithmetic on sparse data via a systolic processing unit. One embodiment provides techniques to optimize training and inference on a systolic array when using sparse data. One embodiment provides techniques to use decompression information when performing sparse compute operations. One embodiment enables the disaggregation of special function compute arrays via a shared reg file. One embodiment enables packed data compress and expand operations on a GPGPU. One embodiment provides techniques to exploit block sparsity within the cache hierarchy of a GPGPU.

Systolic disaggregation within a matrix accelerator architecture

Methods and apparatus relating to scalar core integration in a graphics processor. In an example, an apparatus comprises a processor to receive a set of workload instructions for a graphics workload from a host complex, determine a first subset of operations in the set of operations that is suitable for execution by a scalar processor complex of the graphics processing device and a second subset of operations in the set of operations that is suitable for execution by a vector processor complex of the graphics processing device, assign the first subset of operations to the scalar processor complex for execution to generate a first set of outputs, assign the second subset of operations to the vector processor complex for execution to generate a second set of outputs. Other embodiments are also disclosed and claimed.

Scalar core integration

Growing community of inventors

Portland, OR, United States of America

Mike MacPherson