Intel Corporation

Tatiana Shpeisman

Xiaoming Chen

Ping Tak Tang

Joydeep Ray

Anbang Yao

Michael S Strickland

Abhishek R Appu

Altug Koker

Rajkishore Barik

Ali-Reza Adl-Tabatabai

Eriko Nurvitadhi

Balaji Vembu

Vasanth Ranganathan

Sanjeev S Jahagirdar

Narayan Srinivasa

Feng Chen

Dukhwan Kim

Nadathur Rajagopalan Satish

Farshad Akhbari

Linda L Hurd

Kamal Sinha

Nicolas C Galoppo Von Borries

Tsung-Han Lin

Elmoustapha Ould-Ahmed-Vall

Ben J Ashbaugh

Mike B Macpherson

Liwei Ma

Sanu K Mathew

John Charles Weast

Mark A Anders

Himanshu Kaul

Barath Lakshmanan

Vijay Menon

Justin E Gottschlich

Kevin Nealis

Jeremy Bottleson

Dhawal Srivastava

Guei-Yuan Lueh

Bratin Saha

Brian T Lewis

Irina Calciu

Richard Lewis Hudson

Yang Ni

Brian Murphy

Cheng C Wang

Sara S Baghsorkhi

Narayan Biswal

Stanley J Baran

Gokcen Cilingir

Nilesh V Shah

Tomasz Janczak

Barnan Das

Archie Sharma

Gilles A Pokam

Mayuresh M Varerkar

Sherine Abdelhak

Jayanth N Rao

Jingyi Jin

Sachin B Godse

Stephan A Herhut

Jayashankar Bharadwaj

Jaswanth Sreeram

Rashid Kaleem

Youfeng Wu

Victor W Lee

Mike MacPherson

Paul M Petersen

Arch D Robison

Hongbo Rong

Todd A Anderson

Cheng-Hsueh Andrew Hsieh

Michal J Cierniak

Peng Tu

Leaf Petersen

Brian R Murphy

Chunling Hu

Ali-Reza Adl-Tabataba

P G Lowney

Hai Liu

Described herein is a general-purpose graphics processing unit including a multiprocessor having a single instruction, multiple thread, SIMT, architecture. The multiprocessor comprises multiple sets of compute units each having a first logic unit configured to perform floating-point operations and a second logic unit configured to perform integer operations, with a thread of the floating-point instruction being executed in parallel with a thread of the integer instruction.

Multicore processor with each core having independent floating point datapath and integer datapath

In an example, an apparatus comprises a compute engine comprising a high precision component and a low precision component; and logic, at least partially including hardware logic, to receive instructions in the compute engine; select at least one of the high precision component or the low precision component to execute the instructions; and apply a gate to at least one of the high precision component or the low precision component to execute the instructions. Other embodiments are also disclosed and claimed.

Dynamic precision for neural network compute operations

One embodiment provides for a graphics processing unit to accelerate machine-learning operations, the graphics processing unit comprising a multiprocessor having a single instruction, multiple thread (SIMT) architecture, the multiprocessor to execute at least one single instruction; and a first compute unit included within the multiprocessor, the at least one single instruction to cause the first compute unit to perform a two-dimensional matrix multiply and accumulate operation, wherein to perform the two-dimensional matrix multiply and accumulate operation includes to compute an intermediate product of 16-bit operands and to compute a 32-bit sum based on the intermediate product.

Instructions and logic to perform floating point and integer operations for machine learning

One embodiment provides for a graphics processing unit (GPU) to accelerate machine learning operations, the GPU comprising an instruction cache to store a first instruction and a second instruction, the first instruction to cause the GPU to perform a floating-point operation, including a multi-dimensional floating-point operation, and the second instruction to cause the GPU to perform an integer operation; and a general-purpose graphics compute unit having a single instruction, multiple thread architecture, the general-purpose graphics compute unit to concurrently execute the first instruction and the second instruction.

Concurrent multi-datatype execution within a processing resource

Methods, apparatus, systems, and articles of manufacture are disclosed to steal work in heterogeneous computing systems. An apparatus includes load balancing circuitry to obtain tasks from a workload by encoding minimum and maximum index ranges of a data parallel operation, allocate a first task from the workload to a first work queue based on a first capability of first computation circuitry, the first computation circuitry to process the first task in the first work queue, and allocate a second task from the workload to a second work queue, second computation circuitry to process the second task in the second work queue. The apparatus further includes first work stealer logic to steal the second task from the second work queue using an atomic operation to access the second work queue.

Work stealing in heterogeneous computing systems

One embodiment provides for a graphics processing unit to accelerate machine-learning operations, the graphics processing unit comprising a multiprocessor having a single instruction, multiple thread (SIMT) architecture, the multiprocessor to execute at least one single instruction; and a first compute unit included within the multiprocessor, the at least one single instruction to cause the first compute unit to perform a two-dimensional matrix multiply and accumulate operation, wherein to perform the two-dimensional matrix multiply and accumulate operation includes to compute a 32-bit intermediate product of 16-bit operands and to compute a 32-bit sum based on the 32-bit intermediate product.

An apparatus to facilitate compute optimization is disclosed. The apparatus includes a mixed precision core including mixed-precision execution circuitry to execute one or more of the mixed-precision instructions to perform a mixed-precision dot-product operation comprising to perform a set of multiply and accumulate operations.

Compute optimization mechanism

One embodiment provides for a general-purpose graphics processing unit including a scheduler to schedule multiple matrix operations for execution by a general-purpose graphics processing unit. The multiple matrix operations are determined based on a single machine learning compute instruction. The single machine learning compute instruction is a convolution instruction and the multiple matrix operations are associated with a convolution operation.

Specialized fixed function hardware for efficient convolution

A library of machine learning primitives is provided to optimize a machine learning model to improve the efficiency of inference operations. In one embodiment a trained convolutional neural network (CNN) model is processed into a trained CNN model via pruning, convolution window optimization, and quantization.

Convolutional neural network optimization mechanism

An apparatus to facilitate optimization of a convolutional neural network (CNN) is disclosed. The apparatus includes optimization logic to receive a CNN model having a list of instructions and including pruning logic to optimize the list of instructions by eliminating branches in the list of instructions that comprise a weight value of 0.

An integrated circuit (IC) package apparatus is disclosed. The IC package includes one or more processing units and a bridge, mounted below the one or more processing unit, including one or more arithmetic logic units (ALUs) to perform atomic operations.

Graphics processing integrated circuit package

Embodiments provide systems and methods which facilitate optimization of a convolutional neural network (CNN). One embodiment provides for a non-transitory machine-readable medium storing instructions that cause one or more processors to perform operations comprising processing a trained convolutional neural network (CNN) to generate a processed CNN, the trained CNN having weights in a floating-point format. Processing the trained CNN includes quantizing the weights in the floating-point format to generate weights in an integer format. Quantizing the weights includes generating a quantization table to enable non-uniform quantization of the weights and quantizing the weights from the floating-point format to the integer format using the quantization table. The operations additionally comprise performing an inference operation utilizing the processed CNN with the integer format weights.

One embodiment provides a graphics processor comprising a memory controller and a graphics processing resource coupled with the memory controller. The graphics processing resource includes circuitry configured to execute an instruction to perform a matrix operation on first input including weight data and second input including input activation data, generate intermediate data based on a result of the matrix operation, quantize the intermediate data to a floating-point format determined based on a statistical distribution of first output data, and output, as second output data, quantized intermediate data in a determined floating-point format.

Growing community of inventors

Menlo Park, CA, United States of America

Tatiana Shpeisman