Samsung Electronics Co., Ltd.

Korea Advanced Institute of Science and Technology

Ulsan National Institute of Science and Technology

Unist (ulsan National Institute of Science and Technology )

Sehwan Lee

Joonho Song

Junwoo Jang

Jun-Woo Jang

Yibing Michelle Wang

Ilia Ovsiannikov

Joseph H Hassoun

Ali Shafiee Ardestani

Lei Wang

Yuecheng Li

Namjoon Kim

Ankur Deshwal

Kiran Kolar Chandrasekharan

Saptarsi Das

Hyunsun Park

Pramod Parameshwara Udupa

Seungwon Lee

Yoojin Kim

Jongeun Lee

Jaehyeong Sim

Leesup Kim

Hyeonuk Kim

Jinook Song

Deokjin Joo

Yeongjae Choi

Arnab Roy

Gopinath Vasanth Mahale

Kiran Kolar Chandrasek Haran

Dinesh Kumar Yadav

Sabitha Kusuma

Junseok Park

Jong Hwa Kim

Zhi-Gang Liu

Hyeongseok Yu

Junhaeng Lee

Wonjo Lee

Channoh Kim

Hyun Sun Park

SeungWon Lee

Hyeon Uk Sim

Jooyeon Choi

Daekyeung Kim

Yunkyo Cho

Jun Woo Jang

Dongkyun Kim

An integrated circuit includes: a central processing unit (CPU) core; an accelerator; and an acceleration instruction queue connected to the CPU core and the accelerator. The CPU core is to: fetch and decode one or more instructions from among an instruction sequence in a programmed order; determine an instruction from among the one or more instructions containing an acceleration workload encoded therein; and queue the instruction containing the acceleration workload encoded therein in the acceleration instruction queue.

CPU tight-coupled accelerator

Disclosed is a hybrid traversal apparatus and method for a convolution neural network (CNN) accelerator architecture that receives input feature map (IFM) microbatches from a pixel memory and receiving kernel microbatches from a kernel memory, multiplies the IFM microbatches by the kernel microbatches while reusing the kernel microbatches based on a kernel reuse factor for at least one of a direct convolution (DConv) or a Winograd convolution (WgConv), to obtain output feature map (OFM) microbatches, and writes the generated OFM microbatches to the pixel memory, after quantization, non-linear function, and pooling on a result of the multiplying.

Power-efficient hybrid traversal apparatus and method for convolutional neural network accelerator architecture

A z-first reference neural processing unit (NPU) for mapping Winograd Convolution is disclosed where the NPU includes memory banks configured to store input feature maps (IFMs) in a z-first data storage layout, each of the memory banks being configured to store the IFMs in one of a direct convolution (DConv) mode or a Winograd convolution (WgConv) mode, a reconfigurable IFM distributor configured to receive the IFMs from the memory banks, a parallel reconfigurable Winograd forward transform module configured to receive the IFMs from the reconfigurable IFM distributor and to transform the IFMs in a Winograd domain to transformed IFMs in the WgConv mode, multiply and accumulate (MAC) units configured to perform dot product operations on one of IFMs in the DConv mode and the transformed IFMs in the WgConv mode to obtain intermediate output feature maps (OFMs), and a reconfigurable OFM adder and Winograd inverse transform module configured to generate one of an OFM from the intermediate OFMs in the DConv mode and OFMs from the intermediate OFMs in the WgConv.

Z-first reference neural processing unit for mapping winograd convolution and a method thereof

A processor-implemented neural network operation method includes: receiving a first activation gradient and a first threshold corresponding to a layer included in a neural network; sparsifying the first activation gradient based on the first threshold; determining a second activation gradient by performing a neural network operation based on the sparsified first activation gradient; determining a second threshold by updating the first threshold based on the second activation gradient; and performing a neural network operation based on the second activation gradient and the second threshold.

Method and apparatus with neural network operation using sparsification

A processor-implemented method for generating Output Feature Map (OFM) channels using a Convolutional Neural Network (CNN), include a plurality of kernels, includes generating at least one encoded Similar or Identical Inter-Kernel Weight (S/I-IKW) stream, converting, similar and identical weights in the at least one non-pivot kernel to zero to introduce sparsity into the at least one non-pivot kernel, broadcasting at least one value to the at least one non-pivot kernel, and generating at least one OFM channel by accumulating an at least one previous OFM value with any one or any combination of any two or more of a convolution of non-zero weights of the pivot kernel and pixels of the Input Feature Map (IFM), the at least one broadcasted value, and a convolution of non-zero weights of the at least one non-pivot kernel and pixels of the IFM.

Methods and systems with convolutional neural network (CNN) performance

A neural network apparatus including one or more processors including a controller configured to determine a shared operand to be shared in parallelized operations as being either one of a pixel value among pixel values of an input feature map and a weight value among weight values of a kernel, based on either one or both of a feature of the input feature map and a feature of the kernel, and one or more processing units configured to perform the parallelized operations based on the determined shared operand.

Method and apparatus with convolution neural network processing using shared operand

A method for performing a pooling operation in bitwise manner, the method includes performing a pooling operation on ternary data upon receiving an input ternary vector, receiving an input binary vector, providing a fused hardware for performing the pooling operation on any of the received binary and the ternary data, and executing the pooling operation performed bitwise through the fused hardware.

Methods and systems with bitwise pooling operations

A neural network operation apparatus includes an input register to store an input feature map, a processing element array including a processing element to perform an operation based on the input feature map and a weight matrix, and a controller to map a portion of the input feature map and a portion of the weight matrix, both on which the operation is to be performed, to the processing element.

Neural network operation method and apparatus with mapping orders

A neural processor. In some embodiments, the processor includes a first tile, a second tile, a memory, and a bus. The bus may be connected to the memory, the first tile, and the second tile. The first tile may include: a first weight register, a second weight register, an activations buffer, a first multiplier, and a second multiplier. The activations buffer may be configured to include: a first queue connected to the first multiplier and a second queue connected to the second multiplier. The first queue may include a first register and a second register adjacent to the first register, the first register being an output register of the first queue. The first tile may be configured: in a first state: to multiply, in the first multiplier, a first weight by an activation from the output register of the first queue, and in a second state: to multiply, in the first multiplier, the first weight by an activation from the second register of the first queue.

Neural processor

A neural network apparatus includes a plurality of node buffers connected to a node lane and configured to store input node data by a predetermined bit size; a plurality of weight buffers connected to a weight lane and configured to store weights; and one or more processors configured to: generate first and second split data by splitting the input node data by the predetermined bit size, store the first and second split data in the node buffers, output the first split data to an operation circuit for a neural network operation on an index-by-index basis, shift the second split data, and output the second split data to the operation circuit on the index-by-index basis.

Apparatus and method for processing convolution operation of neural network

A neural processing device includes a first memory configured to store universal data, a second memory distinguished from the first memory and having a capacity less than that of the first memory, a bandwidth control path configured to reconfigure a memory bandwidth for memory clients to use one of the first memory and the second memory based on a control signal, and a control logic configured to calculate a target capacity for data of a target client of the memory clients determined based on a layer configuration of an artificial neural network, and generate the control signal to store the data of the target client in the second memory based on a result of comparing the target capacity and the capacity of the second memory.

Neural processing device and operation method of the neural processing device

An arithmetic apparatus includes a first operand holding circuit configured to output a first operand according to a clock signal, generate an indicator signal based on bit values of high-order bit data including a most significant bit of the first operand, and gate the clock signal based on the indicator signal, the clock signal being applied to a flip-flop latching the high-order bit data of the first operand; a second operand holding circuit configured to output a second operand according to the clock signal; and an arithmetic circuit configured to perform data gating on the high-order bit data of the first operand based on the indicator signal and output an operation result by performing an operation using a modified first operand resulting from the data gating and the second operand.

Arithmetic apparatus, operating method thereof, and neural network processor

Growing community of inventors

Suwon-si, South Korea

Sehwan Lee