Intel Corporation

Subramaniam M Maiyuran

Abhishek R Appu

Altug Koker

Joydeep Ray

Prasoonkumar Surti

Guei-Yuan Lueh

Varghese George

Elmoustapha Ould-Ahmed-Vall

Balaji Vembu

Vasanth Ranganathan

Valentin Andrei

Murali Ramadoss

Aravindh V Anantaraman

Mike MacPherson

Kamal Sinha

Travis T Schluessler

Nicolas C Galoppo Von Borries

Devan Burke

David M Puffer

Ben J Ashbaugh

Lakshminarayanan Striramassarma

Sanjeev S Jahagirdar

Adam T Lake

Peter L Doyle

John H Feit

Karthik Vaidyanathan

Nikos Kaburlasos

Linda L Hurd

Kiran Chand Veernapu

Scott R Janus

Philip R Laws

Jeffery S Boles

Wenyin Fu

Josh B Mastronarde

Pattabhiraman K

Narayan Srinivasa

Justin E Gottschlich

Ankur N Shah

Feng Chen

Eriko Nurvitadhi

Sara S Baghsorkhi

Rajkishore Barik

Dukhwan Kim

Nadathur Rajagopalan Satish

James M Holland

Tsung-Han Lin

Farshad Akhbari

Shekoufeh Qawami

Stephen Anthony Fischer

James A Valerio

Niranjan L Cooray

Michael Apodaca

Lidong Xu

Kun Tian

Eric C Samson

Aditya Navale

Gabor Liktor

Vladimir M Pentkovski

Abhishek Venkatesh

Jacek Kwiatkowski

Dipankar Das

Zhi Hu Wang

Thomas A Piazza

Ingo Wald

Vivek Garg

Jonathan Kennedy

Naveen K Mellempudi

Jill MacDonald Boyce

Rajesh M Sankaran

Tomer Bar-On

Hugues Labbe

John G Gierach

Christopher J Hughes

Sanjay Dabral

Kai Xiao

Chandrasekaran Sakthivel

Daniel Pohl

Bhushan M Borole

Bret L Toll

Muhammad M Khellah

Tomas G Akenine-Moller

James W Tschanz

Changliang Wang

Zhi Jun Wang

Robert M Toth

Jin Xu

Mohammad A Abdallah

Hong Jiang

Alexander F Heinecke

Stephan Jourdan

Shreekant S Thakkar

Jon N Hasselgren

Carsten Benthin

Carl Jacob Munkberg

Selvakumar Panneer

Roger A Golliver

Dan Baum

Sven Woop

Mike B Macpherson

Lakshminarayana Pappu

Embodiments described herein are generally directed to improvements relating to power, latency, bandwidth and/or performance issues relating to GPU processing/caching. According to one embodiment, a system includes a producer intellectual property (IP) (e.g., a media IP), a compute core (e.g., a GPU or an AI-specific core of the GPU), a streaming buffer logically interposed between the producer IP and the compute core. The producer IP is operable to consume data from memory and output results to the streaming buffer. The compute core is operable to perform AI inference processing based on data consumed from the streaming buffer and output AI inference processing results to the memory.

Graphics processing unit processing and caching improvements

One embodiment provides a graphics processor including a processing resource including a register file, memory, a cache memory, and load/store/cache circuitry to process load, store, and prefetch messages from the processing resource. The circuitry includes support for an immediate address offset that will be used to adjust the address supplied for a memory access to be requested by the circuitry. Including support for the immediate address offset removes the need to execute additional instructions to adjust the address to be accessed prior to execution of the memory access instruction.

Immediate offset of load store and atomic instructions

Dynamic tile sequencing in graphics processing is described. An example of an apparatus includes one or more processors including a graphics processor, the one or more processor including a plurality of portions and tile sequencing circuitry; and a memory to store data for graphics processing, including data for a render target and data for a hashing table, the render target including a plurality of tiles, and the hashing table to map the tiles of the render target to the plurality of portions of the one or more processors, wherein the tile sequencing circuitry includes a first mode for tile sequencing, wherein tile sequencing in the first mode includes a set granularity for the hashing table; and a second mode for tiling sequencing, wherein tile sequencing in the second mode includes a configurable granularity for the hashing table.

Dynamic tile sequencing in graphic processing

In accordance with some embodiments, the render rate is varied across and/or up and down the display screen. This may be done based on where the user is looking in order to reduce power consumption and/or increase performance. Specifically the screen display is separated into regions, such as quadrants. Each of these regions is rendered at a rate determined by at least one of what the user is currently looking at, what the user has looked at in the past and/or what it is predicted that the user will look at next. Areas of less focus may be rendered at a lower rate, reducing power consumption in some embodiments.

Regional adjustment of render rate

Embodiments described herein provide an apparatus comprising a plurality of processing resources including a first processing resource and a second processing resource, a shared local memory communicatively coupled to the first processing resource and the second processing resource, and a processor to receive an instruction to initiate a matrix multiplication operation, write a first set of matrix data into a first set of registers, and share the first set of matrix data between the first processing resource and the second processing resource for use in the matrix multiplication operation. Other embodiments may be described and claimed.

Sharing register file usage between fused processing resources

An apparatus to facilitate utilizing structured sparsity in systolic arrays is disclosed. The apparatus includes a processor comprising a systolic array to receive data from a plurality of source registers, the data comprising unpacked source data, structured source data that is packed based on sparsity, and metadata corresponding to the structured source data; identify portions of the unpacked source data to multiply with the structured source data, the portions of the unpacked source data identified based on the metadata; and output, to a destination register, a result of multiplication of the portions of the unpacked source data and the structured source data.

Utilizing structured sparsity in systolic arrays

In an example, an apparatus comprises a plurality of execution units, and a first memory communicatively couple to the plurality of execution units, wherein the first shared memory is shared by the plurality of execution units and a copy engine to copy context state data from at least a first of the plurality of execution units to the first shared memory. Other embodiments are also disclosed and claimed.

Engine to enable high speed context switching via on-die storage

A processing apparatus is described herein that includes a general-purpose parallel processing engine comprising a matrix accelerator including one or more systolic arrays, at least one of the one or more systolic arrays comprising multiple pipeline stages, each pipeline stage of the multiple pipeline stages including multiple processing elements, the multiple processing elements configured to perform processing operations on input matrix elements based on output sparsity metadata. The output sparsity metadata indicates to the multiple processing elements to bypass multiplication for a first row of elements of a second matrix and multiply a second row of elements of the second matrix with a column of matrix elements of a first matrix.

Systolic array having support for output sparsity

Embodiments described herein provide techniques to enable the dynamic reconfiguration of memory on a general-purpose graphics processing unit. One embodiment described herein enables dynamic reconfiguration of cache memory bank assignments based on hardware statistics. One embodiment enables for virtual memory address translation using mixed four kilobyte and sixty-four kilobyte pages within the same page table hierarchy and under the same page directory. One embodiment provides for a graphics processor and associated heterogenous processing system having near and far regions of the same level of a cache hierarchy.

Dynamic memory reconfiguration

An apparatus to facilitate gathering payload from arbitrary registers for send messages in a graphics environment is disclosed. The apparatus includes processing resources comprising execution circuitry to receive a send gather message instruction identifying a number of registers to access for a send message and identifying IDs of a plurality of individual registers corresponding to the number of registers; decode a first phase of the send gather message instruction; based on decoding the first phase, cause a second phase of the send gather message instruction to bypass an instruction decode stage; and dispatch the first phase subsequently followed by dispatch of the second phase to a send pipeline. The apparatus can also perform an immediate move of the IDs of the plurality of individual registers to an architectural register of the execution circuitry and include a pointer to the architectural register in the send gather message instruction.

Gathering payload from arbitrary registers for send messages in a graphics environment

Embodiments are generally directed to memory prefetching in multiple GPU environment. An embodiment of an apparatus includes multiple processors including a host processor and multiple graphics processing units (GPUs) to process data, each of the GPUs including a prefetcher and a cache; and a memory for storage of data, the memory including a plurality of memory elements, wherein the prefetcher of each of the GPUs is to prefetch data from the memory to the cache of the GPU; and wherein the prefetcher of a GPU is prohibited from prefetching from a page that is not owned by the GPU or by the host processor.

Prefetch status notification for memory prefetching

Embodiments described herein provided for an instruction and associated logic to enable a processing resource including a tensor accelerator to perform optimized computation of sparse submatrix operations. One embodiment provides a parallel processor comprising a processing cluster coupled with the cache memory. The processing cluster includes a plurality of multiprocessors coupled with a data interconnect, where a multiprocessor of the plurality of multiprocessors includes a tensor core configured to load tensor data and metadata associated with the tensor data from the cache memory, wherein the metadata indicates a first numerical transform applied to the tensor data, perform an inverse transform of the first numerical transform, perform a tensor operation on the tensor data after the inverse transform is performed, and write output of the tensor operation to a memory coupled with the processing cluster.

Systolic arithmetic on sparse data

A processing apparatus includes a general-purpose parallel processing engine including a set of multiple processing elements including a single precision floating-point unit, a double precision floating point unit, and an integer unit; a matrix accelerator including one or more systolic arrays; a first register file coupled with a first read control circuit, wherein the first read control circuit couples with the set of multiple processing elements and the matrix accelerator to arbitrate read requests to the first register file from the set of multiple processing elements and the matrix accelerator; and a second register file coupled with a second read control circuit, wherein the second read control circuit couples with the matrix accelerator to arbitrate read requests to the second register file from the matrix accelerator and limit access to the second register file by the set of multiple processing elements.

Register file for systolic array

A graphics processing apparatus includes a graphics processor and a constant cache. The graphics processor has a number of execution instances that will generate requests for constant data from the constant cache. The constant cache stores constants of multiple constant types. The constant cache has a single level of hierarchy to store the constant data. The constant cache has a banking structure based on the number of execution instances, where the execution instances generate requests for the constant data with unified messaging that is the same for the different types of constant data.

High performance constant cache and constant access mechanisms

Embodiments described herein include, software, firmware, and hardware logic that provides techniques to perform arithmetic on sparse data via a systolic processing unit. Embodiment described herein provided techniques to detect zero value elements within a vector or a set of packed data elements output by a processing resource and generate metadata to indicate a location of the zero value elements within the plurality of data elements.

Sparse optimizations for a matrix accelerator architecture

Growing community of inventors

Gold River, CA, United States of America

Subramaniam M Maiyuran