Intel Corporation

Other

Guei-Yuan Lueh

Subramaniam M Maiyuran

Altug Koker

Wei-Yu Chen

Abhishek R Appu

Prasoonkumar Surti

Joydeep Ray

Balaji Vembu

Supratim Pal

Chandra S Gurram

David M Puffer

Jorge E Parra

Gansha Wu

Murali Ramadoss

Ashutosh Garg

Travis T Schluessler

Varghese George

Darin Starkey

Ali-Reza Adl-Tabatabai

Vasanth Ranganathan

Kamal Sinha

Nikos Kaburlasos

Linda L Hurd

Kiran Chand Veernapu

James A Valerio

Arthur Hunter, Jr

Eric J Hoekstra

Shubra Marwaha

Hong Jiang

Tatiana Shpeisman

Ingo Wald

Christopher J Hughes

Sanjeev S Jahagirdar

Adam T Lake

Lidong Xu

Gabor Liktor

Tomer Bar-On

Hugues Labbe

John G Gierach

Xinmin Tian

Andrew T Lauritzen

Slawomir Grajewski

Xin Zhou

Kai Xiao

Changliang Wang

James M Holland

Abhishek Venkatesh

Wenyin Fu

Josh B Mastronarde

Bhushan M Borole

Stephen Junkins

Tomasz Janczak

David J Cowperthwaite

Vikranth Vemulapalli

Peng Guo

Lei Shen

Eric J Asperheim

Timothy R Bauer

Elmoustapha Ould-Ahmed-Vall

Jill MacDonald Boyce

Hong Wang

Ankur N Shah

Kun Tian

Eric C Samson

Gautham N Chinya

Devan Burke

Dipankar Das

Mike MacPherson

Jonathan Kennedy

Jonathan D Pearce

Thomas F Raoux

Brian T Lewis

Jorge F Garcia Pabon

Kalyan Kumar Bhiravabhatla

Ping Liu

Anwar Ghuloum

Robert Valentine

Yang Liu

Mark Jay Charney

Zhi Hu Wang

Tomas G Akenine-Moller

Alexander F Heinecke

Michael Apodaca

Raanan Sade

Scott R Janus

Yong Jiang

Zhi Jun Wang

John Thomas Shen

Dan Baum

Richard A Hankins

Perry H Wang

Mike B Macpherson

Brent E Insko

Ronen Zohar

Fangwen Fu

Trung Am Diep

Michael J Espig

Jamison D Collins

Paul M Petersen

Sanjiv M Shah

Embodiments described herein are generally directed to improvements relating to power, latency, bandwidth and/or performance issues relating to GPU processing/caching. According to one embodiment, a system includes a producer intellectual property (IP) (e.g., a media IP), a compute core (e.g., a GPU or an AI-specific core of the GPU), a streaming buffer logically interposed between the producer IP and the compute core. The producer IP is operable to consume data from memory and output results to the streaming buffer. The compute core is operable to perform AI inference processing based on data consumed from the streaming buffer and output AI inference processing results to the memory.

Graphics processing unit processing and caching improvements

Examples described herein relate to computer-readable medium comprising instructions stored thereon, that if executed by one or more processors, cause the one or more processors to: access a kernel source written in a shading language; select a compiler from a plurality of compilers based on the shading language; generate a kernel instruction set architecture and an associated runtime binding based on the kernel source and the selected compiler; and adapt state information to configure a graphics processing unit (GPU) based on the runtime binding to target any API runtime of choice.

Kernel source adaptation for execution on a graphics processing unit

In an example, an apparatus comprises a plurality of execution units, and a first memory communicatively couple to the plurality of execution units, wherein the first shared memory is shared by the plurality of execution units and a copy engine to copy context state data from at least a first of the plurality of execution units to the first shared memory. Other embodiments are also disclosed and claimed.

Engine to enable high speed context switching via on-die storage

An apparatus to facilitate gathering payload from arbitrary registers for send messages in a graphics environment is disclosed. The apparatus includes processing resources comprising execution circuitry to receive a send gather message instruction identifying a number of registers to access for a send message and identifying IDs of a plurality of individual registers corresponding to the number of registers; decode a first phase of the send gather message instruction; based on decoding the first phase, cause a second phase of the send gather message instruction to bypass an instruction decode stage; and dispatch the first phase subsequently followed by dispatch of the second phase to a send pipeline. The apparatus can also perform an immediate move of the IDs of the plurality of individual registers to an architectural register of the execution circuitry and include a pointer to the architectural register in the send gather message instruction.

Gathering payload from arbitrary registers for send messages in a graphics environment

A processing apparatus includes a general-purpose parallel processing engine including a set of multiple processing elements including a single precision floating-point unit, a double precision floating point unit, and an integer unit; a matrix accelerator including one or more systolic arrays; a first register file coupled with a first read control circuit, wherein the first read control circuit couples with the set of multiple processing elements and the matrix accelerator to arbitrate read requests to the first register file from the set of multiple processing elements and the matrix accelerator; and a second register file coupled with a second read control circuit, wherein the second read control circuit couples with the matrix accelerator to arbitrate read requests to the second register file from the matrix accelerator and limit access to the second register file by the set of multiple processing elements.

Register file for systolic array

A graphics processing apparatus includes a graphics processor and a constant cache. The graphics processor has a number of execution instances that will generate requests for constant data from the constant cache. The constant cache stores constants of multiple constant types. The constant cache has a single level of hierarchy to store the constant data. The constant cache has a banking structure based on the number of execution instances, where the execution instances generate requests for the constant data with unified messaging that is the same for the different types of constant data.

High performance constant cache and constant access mechanisms

Provision of multiple register allocation sizes for threads is described. An example of a system includes one or more processors including a graphics processor, the graphics processor including at least a first local thread dispatcher (TDL) and multiple processing resources, each processing resource including a plurality of registers; and memory for storage of data for processing, wherein the one or more processors are to determine a register size for a first thread; identify one or more processing resources having sufficient register space for the first thread; select a processing resource of the one or more processing resources having sufficient register space to assign the first thread; select an available thread slot of the selected processing resource for the first thread; and allocate registers of the selected processing resource for the first thread.

Multiple register allocation sizes for threads

Embodiments described herein provide an apparatus comprising a plurality of processing resources including a first processing resource and a second processing resource, a memory communicatively coupled to the first processing resource and the second processing resource, and a processor to receive data dependencies for one or more tasks comprising one or more producer tasks executing on the first processing resource and one or more consumer tasks executing on the second processing resource and move a data output from one or more producer tasks executing on the first processing resource to a cache memory communicatively coupled to the second processing resource. Other embodiments may be described and claimed.

Data locality enhancement for graphics processing units

A processing apparatus includes a processing resource including a general-purpose parallel processing engine and a matrix accelerator. The matrix accelerator includes first circuitry to receive a command to perform operations associated with an instruction, second circuitry to configure the matrix accelerator according to a physical depth of a systolic array within the matrix accelerator and a logical depth associated with the instruction, third circuitry to read operands for the instruction from a register file associated with the systolic array, fourth circuitry to perform operations for the instruction via one or more passes through one or more physical pipeline stages of the systolic array based on a configuration performed by the second circuitry, and fifth circuitry to write output of the operations to the register file associated with the systolic array.

Systolic array of arbitrary physical and logical depth

An apparatus to facilitate data prefetching is disclosed. The apparatus includes a cache, one or more execution units (EUs) to execute program code, prefetch logic to maintain tracking information of memory instructions in the program code that trigger a cache miss and compiler logic to receive the tracking information, insert one or more pre-fetch instructions in updated program code to prefetch data from a memory for execution of one or more of the memory instructions that triggered a cache miss and download the updated program code for execution by the one or more EUs.

Instruction prefetch mechanism

One embodiment provides a graphics processor comprising a system interface and circuitry coupled with the system interface. The circuitry includes an execution resource and a preemption status register. The execution resource is configured to execute an instruction. During execution of the instruction, the execution resource is to receive a request to preempt execution of a thread associated with the instruction and, based on a value stored in the preemption status register, execute at least one additional instruction after receipt of the request to preempt execution of the thread.

Page faulting and selective preemption

One embodiment provides a parallel processor comprising a memory interface and a processing array coupled with the memory interface. The processing array is configured to address memory accessed via the memory interface via a virtual address mapping and includes circuitry to resolve a page fault for the virtual address mapping, wherein each of the multiple compute blocks is separately preemptable.

Graphics processors and graphics processing units having dot product accumulate instructions for a hybrid floating point format are disclosed. In one embodiment, a graphics multiprocessor comprises an instruction unit to dispatch instructions and a processing resource coupled to the instruction unit. The processing resource is configured to receive a dot product accumulate instruction from the instruction unit and to process the dot product accumulate instruction using a bfloat16 number (BF16) format.

Graphics processors and graphics processing units having dot product accumulate instruction for hybrid floating point format

Described herein is a graphics processing unit (GPU) configured to receive an instruction having multiple operands, where the instruction is a single instruction multiple data (SIMD) instruction configured to use a bfloat16 (BF16) number format and the BF16 number format is a sixteen-bit floating point format having an eight-bit exponent. The GPU can process the instruction using the multiple operands, where to process the instruction includes to perform a multiply operation, perform an addition to a result of the multiply operation, and apply a rectified linear unit function to a result of the addition.

Examples described herein relate to a software and hardware optimization that manages scenarios where a write operation to a register is less than an entirety of the register. A compiler detects instructions that make partial writes to the same register, groups such instructions, and provides hints to hardware of the partial write. The execution unit combines the output data for grouped instructions and updates the destination register as single write instead of multiple separate partial writes.

Compiler assisted register file write reduction

Growing community of inventors

San Jose, CA, United States of America

Guei-Yuan Lueh