International Business Machines Corporation

Cornell University

Dharmendra S Modha

John Vernon Arthur

Paul A Merolla

Andrew S Cassidy

Pallab Datta

Steven Kyle Esser

Jun Sawada

Myron Dale Flickner

Rathinakumar Appuswamy

Rodrigo Alvarez-Icaza Rivera

Bryan L Jackson

Alexander Andreopoulos

Brian Taba

Daniel Joseph Friedman

Arnon Amir

Jose A Tierno

Binny Sher Gill

Bipin Rajendran

Hartmut Penner

Jennifer Klamo

Filipp A Akopyan

John Edward Barth, Jr

Leland Chang

Robert K Montoye

Jae-sun Seo

Bernard V Brezzo

Benjamin G Shaw

Subramanian Srikanteswara Iyer

Chung H Lam

Nimrod Megiddo

Daniel Gruhl

Michael S Karasick

Michael T Benhase

William Scott Spangler

Seongwon Kim

Yong Liu

Tapan K Nayak

Charles Jay Alpert

Gi-Joon Nam

Joseph S Hyde, Ii

Rajit Manohar

Alyosha C Molnar

Thomas Charles Jarvis

William P Risk, Iii

Rodrigo Alvarez-Icaza

Anthony Ndirango

Zhuo Li

Matthew Joseph Breitwisch

Bruce McNutt

Norman Jerome Pass

Steve Kyle Esser

Carlos O Otero

Paul Edward McKenney

Jeffrey Thomas Kreulen

Hovey Raymond Strong

Rohit Sudhir Shenoy

Rajagopal Ananthanarayanan

Brian Harry Marcus

Michael Vincent DeBole

Rodrigo Alvarez Icaza Rivera

Michael V DeBole

Sorav Bansal

Inderjit Singh Dhillon

Jeffrey L McKinstry

Rajagopal (Ananth) Ananthanarayanan

Kangguo Cheng

Ali Khakifirooz

Evangelos Stavros Eleftheriou

Roy Daron Cideciyan

Chandrasekhar Narayan

Divyesh Jadav

Renu Tewari

Vikas Krishna

Gregory Sean Corrado

Shivakumar Vaithyanathan

Arjang Hassibi

Sridhar Rajagopalan

John Campbell Scott

Stuart S P Parkin

Roger W Cheek

David Charles Martin

Jorge Campello De Souza

Raghavendra Singh

John E Barth

David J Berg

Pawan Goyal

Radley Wahl Olson

Theodore M Wong

Michael Beyeler

Deepika Bablani

Filipp Akopyan

Steve Esser

Rodrigo Alcarez-Icaza Rivera

Shyamal S Chandra

Zhou Li

Jeffrey Mckinstry

Andrew Stephen Cassidy

Brian Seisho Taba

Carlos Ortega Otero

Networks of distributed neural cores are provided with hierarchical parallelism. In various embodiments, a plurality of neural cores is provided. Each of the plurality of neural cores comprises a plurality of vector compute units configured to operate in parallel. Each of the plurality of neural cores is configured to compute in parallel output activations by applying its plurality of vector compute units to input activations. Each of the plurality of neural cores is assigned a subset of output activations of a layer of a neural network for computation. Upon receipt of a subset of input activations of the layer of the neural network, each of the plurality of neural cores computes a partial sum for each of its assigned output activations, and computes its assigned output activations from at least the computed partial sums.

Hierarchical parallelism in a network of distributed neural network cores

A neural inference chip includes a global weight memory; a neural core; and a network connecting the global weight memory to the at least one neural core. The neural core comprises a local weight memory. The local weight memory comprises a plurality of memory banks. Each of the plurality of memory banks is uniquely addressable by at least one index. The neural inference chip is adapted to store in the global weight memory a compressed weight block comprising at least one compressed weight matrix. The neural inference chip is adapted to transmit the compressed weight block from the global weight memory to the core via the network. The core is adapted to decode the at least one compressed weight matrix into a decoded weight matrix and store the decoded weight matrix in its local weight memory. The at core is adapted to apply the decoded weight matrix to a plurality of input activations to produce a plurality of output activations.

Compressed weight distribution in networks of neural processors

Conflict-free, stall-free, broadcast networks on neural inference chips are provided. In various embodiments, a neural inference chip comprises a plurality of network nodes and a network on chip interconnecting the plurality of network nodes. The network comprises at least one pair of directional paths. The paths of each pair have opposite directions and a common end. The network is configured to accept data at any of the plurality of nodes. The network is configured to propagate data along a first of the pair of directional paths from a source node to the common end of the pair of directional paths and along a second of the pair of directional paths from the common end of the pair of directional paths to one or more destination node.

Conflict-free, stall-free, broadcast network on chip

A neural inference chip is provided, including at least one neural inference core. The at least one neural inference core is adapted to apply a plurality of synaptic weights to a plurality of input activations to produce a plurality of intermediate outputs. The at least one neural inference core comprises a plurality of activation units configured to receive the plurality of intermediate outputs and produce a plurality of activations. Each of the plurality of activation units is configured to apply a configurable activation function to its input. The configurable activation function has at least a re-ranging term and a scaling term, the re-ranging term determining the range of the activations and the scaling term determining the scale of the activations. Each of the plurality of activations units is configured to obtain the re-ranging term and the scaling term from one or more look up tables.

Efficient method for VLSI implementation of useful neural network activation functions

Neurosynaptic core placement is provided. In various embodiments, a description of a neural network is read. The neural network comprises a plurality of nodes. For each one of the plurality of nodes, a graph is generated corresponding to the nodes incident on that one of the plurality of nodes within the neural network. For each one of the plurality of nodes, a bin is selected from a plurality of bins for placement of the one of the plurality of nodes. A placement description is output for the plurality of nodes to the plurality of bins.

Functional synthesis of networks of neurosynaptic cores on neuromorphic substrates

Mapping of neural network layers to physical neural cores is provided. In various embodiments, a neural network description describing a plurality of neural network layers is read. Each of the plurality of neural network layers has an associated weight tensor, input tensor, and output tensor. A plurality of precedence relationships among the plurality of neural network layers is determined. The weight tensor, input tensor, and output tensor of each of the plurality of neural network layers are mapped onto an array of neural cores.

Scheduler for mapping neural networks onto an array of neural cores in an inference processing unit

A neural network may include a set of components. The set of components may have timing requirements and a topological order. The relative timing of each component may be computed and the dependencies of the set of components may be enumerated. Mutable components within the set of components may be identified, and the relative timing of the mutable components may be adjusted to satisfy the timing requirements of each component in the set of components.

Automatic timing resolution among neural network components

Systems for neural network computation are provided. A neural network processor comprises a plurality of neural cores. The neural network processor has one or more processor precisions per activation. The processor is configured to accept data having a processor feature dimension. A transformation circuit is coupled to the neural network processor, and is adapted to: receive an input data tensor having an input precision per channel at one or more features; transform the input data tensor from the input precision to the processor precision; divide the input data into a plurality of blocks, each block conforming to one of the processor feature dimensions; provide each of the plurality of blocks to one of the plurality of neural cores. The neural network processor is adapted to compute, by the plurality of neural cores, output of one or more neural network layers.

Data representation for dynamic precision in neural network cores

Networks and encodings therefor are provided that are adapted to provide increased energy efficiency and speed for convolutional operations. In various embodiments, a neural network comprises a plurality of neural cores. Each of the plurality of neural cores comprises a memory. A network interconnects the plurality of neural cores. The memory of each of the plurality of neural cores comprises at least a portion of a weight tensor. The weight tensor comprising a plurality of weights. Each neural core is adapted to retrieve locally or receive a portion of an input image, apply the portion of the weight tensor thereto, and store locally or send a result therefrom via the network to other of the plurality of neural cores.

Neural hardware accelerator for parallel and distributed tensor computations

Networks for distributing parameters and data to neural network compute cores. In various embodiments, a neural inference chip comprises a plurality of neural cores and at least one network interconnecting the plurality of neural cores. Each of the plurality of neural cores is adapted to apply a plurality of synaptic weights to a plurality of input activations to produce a plurality of output activations. The at least one network is adapted to simultaneously deliver synaptic weights and/or input activations to the plurality of neural cores.

Networks for distributing parameters and data to neural network compute cores

Defect resistant designs for location-sensitive neural network processor arrays are provided. In various embodiments, plurality of neural network processor cores are arrayed in a grid. The grid has a plurality of rows and a plurality of columns. A network interconnects at least those of the plurality of neural network processor cores that are adjacent within the grid. The network is adapted to bypass a defective core of the plurality of neural network processor cores by providing a connection between two non-adjacent rows or columns of the grid, and transparently routing messages between the two non-adjacent rows or columns, past the defective core.

Defect resistant designs for location-sensitive neural network processor arrays

Hardware neural network processors, are provided. A neural core includes a weight memory, an activation memory, a vector-matrix multiplier, and a vector processor. The vector-matrix multiplier is adapted to receive a weight matrix from the weight memory, receive an activation vector from the activation memory, and compute a vector-matrix multiplication of the weight matrix and the activation vector. The vector processor is adapted to receive one or more input vector from one or more vector source and perform one or more vector functions on the one or more input vector to yield an output vector. In some embodiments a programmable controller is adapted to configure and operate the neural core.

Runtime reconfigurable neural network processor core

Neural network processing hardware using parallel computational architectures with reconfigurable core-level and vector-level parallelism is provided. In various embodiments, a neural network model memory is adapted to store a neural network model comprising a plurality of layers. Each layer has at least one dimension and comprises a plurality of synaptic weights. A plurality of neural cores is provided. Each neural core includes a computation unit and an activation memory. The computation unit is adapted to apply a plurality of synaptic weights to a plurality of input activations to produce a plurality of output activations. The computation unit has a plurality of vector units. The activation memory is adapted to store the input activations and the output activations. The system is adapted to partition the plurality of cores into a plurality of partitions based on dimensions of the layer and the vector units.

Parallel computational architecture with reconfigurable core-level and vector-level parallelism

Learned step size quantization in artificial neural network is provided. In various embodiments, a system comprises an artificial neural network and a computing node. The artificial neural network comprises: a quantizer having a configurable step size, the quantizer adapted to receive a plurality of input values and quantize the plurality of input values according to the configurable step size to produce a plurality of quantized input values, at least one matrix multiplier configured to receive the plurality of quantized input values from the quantizer and to apply a plurality of weights to the quantized input values to determine a plurality of output values having a first precision, and a multiplier configured to scale the output values to a second precision. The computing node is operatively coupled to the artificial neural network and is configured to: provide training input data to the artificial neural network, and optimize the configurable step size based on a gradient through the quantizer and the training input data.

Learned step size quantization

Instruction distribution in an array of neural network cores is provided. In various embodiments, a neural inference chip is initialized with core microcode. The chip comprises a plurality of neural cores. The core microcode is executable by the neural cores to execute a tensor operation of a neural network. The core microcode is distributed to the plurality of neural cores via an on-chip network. The core microcode is executed synchronously by the plurality of neural cores to compute a neural network layer.

Instruction distribution in an array of neural network cores

Growing community of inventors

San Jose, CA, United States of America

Dharmendra S Modha