Intel Corporation

Gm Global Technology Operations LLC

Indian Institute of Technology Kharagpur

Gm Globla Technology Operations LLC

Dipankar Das

Naveen K Mellempudi

Srinivas Sridharan

Dheevatsa Mudigere

Karthikeyan Vaidyanathan

Sasikanth Avancha

Abhisek Kundu

Subramaniam M Maiyuran

Varghese George

Alexander F Heinecke

Jorge E Parra

Chandra S Gurram

Dhiraj D Kalamkar

Ashutosh Garg

Arun Kumar

Mrinmay Dutta

Jesus Corbal

Chandrasekaran Sakthivel

Supratim Pal

Mikhail Smelyanskiy

Roger Gramunt

Chunhui Mei

Sudarshan Srinivasan

Mikhail E Smorkalov

Dharma Teja Vooturi

Robert Valentine

Guei-Yuan Lueh

Wei-Yu Chen

Anand Raghunathan

Swagath Venkataramani

Yan Li

Shubra Marwaha

Wei Xiong

Durgesh Borkar

Ashish Ranjan

Ishwar Bhati

Junjie Gu

Purnendu Sinha

Yongsheng Liu

Subarno Banerjee

Bharat Kaul

Martin Langhammer

Mark Jay Charney

Hong Jiang

Gregg William Baeckler

Mike MacPherson

Fangwen Fu

Mark Charney

Duncan Moss

Sabareesh Ganapathy

Partha Pratim Chakrabarti

Vinod Kumar Agrawal

Seetharaman Rajappan

Kristopher Wong

Embodiments described herein provide an apparatus comprising an interconnect switch configured to couple with a plurality of graphics processors via a plurality of point-to-point interconnects and one or more processors including a graphics processor coupled with the interconnect switch via a point-to-point interconnect of the plurality of point-to-point interconnects.

Communication optimizations for distributed machine learning

One embodiment provides for a graphics processing unit to perform computations associated with a neural network, the graphics processing unit comprising a hardware processing unit having a dynamic precision fixed-point unit that is configurable to convert elements of a floating-point tensor to convert the floating-point tensor into a fixed-point tensor.

Dynamic precision management for integer deep learning primitives

An apparatus to facilitate utilizing structured sparsity in systolic arrays is disclosed. The apparatus includes a processor comprising a systolic array to receive data from a plurality of source registers, the data comprising unpacked source data, structured source data that is packed based on sparsity, and metadata corresponding to the structured source data; identify portions of the unpacked source data to multiply with the structured source data, the portions of the unpacked source data identified based on the metadata; and output, to a destination register, a result of multiplication of the portions of the unpacked source data and the structured source data.

Utilizing structured sparsity in systolic arrays

A processing apparatus is described herein that includes a general-purpose parallel processing engine comprising a matrix accelerator including one or more systolic arrays, at least one of the one or more systolic arrays comprising multiple pipeline stages, each pipeline stage of the multiple pipeline stages including multiple processing elements, the multiple processing elements configured to perform processing operations on input matrix elements based on output sparsity metadata. The output sparsity metadata indicates to the multiple processing elements to bypass multiplication for a first row of elements of a second matrix and multiply a second row of elements of the second matrix with a column of matrix elements of a first matrix.

Systolic array having support for output sparsity

One embodiment provides for a method of transmitting data between multiple compute nodes of a distributed compute system, the method comprising creating a global view of communication operations to be performed between the multiple compute nodes of the distributed compute system, the global view created using information specific to a machine learning model associated with the distributed compute system; using the global view to determine a communication cost of the communication operations; and automatically determining a number of network endpoints for use in transmitting the data between the multiple compute nodes of the distributed compute system.

Abstraction layers for scalable distributed machine learning

One embodiment provides for a graphics processing unit including a fabric interface configured to transmit gradient data stored in a memory device of the graphics processing unit according to a pre-defined communication operation. The memory device is a physical memory device shared with a compute block of the graphics processing unit and the fabric interface. The fabric interface automatically transmits the gradient data stored in memory to a second distributed training node based on an address of the gradient data in the memory device.

Hardware implemented point to point communication primitives for machine learning

Described herein is a graphics processor including a processing resource including a multiplier configured to multiply input associated with the instruction at one of a first plurality of bit widths, an adder configured to add a product output from the multiplier with an accumulator value at one of a second plurality of bit widths, and circuitry to select a first bit width of the first plurality of bit widths for the multiplier and a second bit width of the second plurality of bit widths for the adder.

Optimized compute hardware for machine learning operations

Disclosed embodiments relate to instructions for fused multiply-add (FMA) operations with variable-precision inputs. In one example, a processor to execute an asymmetric FMA instruction includes fetch circuitry to fetch an FMA instruction having fields to specify an opcode, a destination, and first and second source vectors having first and second widths, respectively, decode circuitry to decode the fetched FMA instruction, and a single instruction multiple data (SIMD) execution circuit to process as many elements of the second source vector as fit into an SIMD lane width by multiplying each element by a corresponding element of the first source vector, and accumulating a resulting product with previous contents of the destination, wherein the SIMD lane width is one of 16 bits, 32 bits, and 64 bits, the first width is one of 4 bits and 8 bits, and the second width is one of 1 bit, 2 bits, and 4 bits.

Instructions for fused multiply-add operations with variable precision input operands

One embodiment provides for a method of transmitting data between multiple compute nodes of a distributed compute system, the method comprising multi-dimensionally partitioning data of a feature map across multiple nodes for distributed training of a convolutional neural network; performing a parallel convolution operation on the multiple partitions to train weight data of the neural network; and exchanging data between nodes to enable computation of halo regions, the halo regions having dependencies on data processed by a different node.

Data parallelism and halo exchange for distributed machine learning

One embodiment provides for a computer-readable medium storing instructions that cause one or more processors to perform operations comprising determining a per-layer scale factor to apply to tensor data associated with layers of a neural network model and converting the tensor data to converted tensor data. The tensor data may be converted from a floating point datatype to a second datatype that is an 8-bit datatype. The instructions further cause the one or more processors to generate an output tensor based on the converted tensor data and the per-layer scale factor.

Incremental precision networks using residual inference and fine-grain quantization

One embodiment provides for a machine-learning accelerator device a multiprocessor to execute parallel threads of an instruction stream, the multiprocessor including a compute unit, the compute unit including a set of functional units, each functional unit to execute at least one of the parallel threads of the instruction stream. The compute unit includes compute logic configured to execute a single instruction to scale an input tensor associated with a layer of a neural network according to a scale factor, the input tensor stored in a floating-point data type, the compute logic to scale the input tensor to enable a data distribution of data of the input tensor to be represented by a 16-bit floating point data type.

Scaling half-precision floating point tensors for training deep neural networks

Growing community of inventors

Pune, India

Dipankar Das