Cadence Design Systems, Inc.

Tabula, Inc.

Perceive Corporation

Adeia Semiconductor Bonding Technologies Inc.

Altera Corporation

Amazon Technologies, Inc.

Other

Adeia Semiconductor Inc.

Altera Coporation

Tbula, Inc.

Mcc - Molecular Simulations

Adence Design Systems, Inc.

Percieve Corporation

Altera Coroporation

Steven L Teig

Andrew Edward Caldwell

Herman Henry Schmit

Brad Hutchings

Jason Rupert Redgrave

Kenneth Duong

Jung Ko

Eric A Sather

Javier A DeLaCruz

Ilyas Mohammed

Andre Rohe

Andrew C Mihal

Joseph L Ganley

Arindam Chatterjee

Randy Renfu Huang

Etienne Jacques

Michael Butts

Akira Fujimura

Asmus Hetzel

Jonathan Frankle

Oscar Buset

Martin L Voogel

Teju Khubchandani

Tom Kronmiller

Brian Thomas

William C Plants

Eric Nequist

David Edward Fisch

Shaowu Huang

Andrea Olgiati

Matthew Pond Baker

David Overhauser

Andrew F Siegel

Raghu Prasad Chalasani

Christopher D Ebeling

Dai Huang

Heng-Yi Chao

Jingcao Hu

Scott J Weber

Daniel J Pugh

Christine M Tarby

Trevis Chandler

Minoru Imai

Shangzhi Sun

Takaharu Tsutsumi

Tatsuki Shiota

Wilna J Moree

Noriaki Endo

Marc A Miller

Osami Takenouchi

Minoru Furuya

Hung Q Nguyen

Masaki Sudoh

Yumiko Muroga

Takahiko Hada

Takuya Morita

Shilpa V Yeole

Timothy J Callahan

Ryo Sogawa

Ken-ichiro Kataoka

Alexandru F Drimbarean

Ryan J Cassidy

Laura Wills Mirkarimi

Louis K Scheffer

Don Draper

Danny Thom

Hussein Etawil

Won Rhee

Scott D Kahn

Thomas S Chanack

Vikas Chandra

Amit Gupta

Javier A Delacruz

Hengfu Hsu

Yang-Trung Lin

Timothy Horel

Serdar Kizilgul

Philip Sharos

Evgeny Sorkin

Murali Dharan

Justin Tantiongloc

Maogang Wang

Zachary Deretsky

Teju Khubehandani

Some embodiments provide a neural network inference circuit for executing a neural network that includes multiple nodes that use state data from previous executions of the neural network. The neural network inference circuit includes (i) a set of computation circuits configured to execute the nodes of the neural network and (ii) a set of memories configured to implement a set of one or more registers to store, while executing the neural network for a particular input, state data generated during at least two executions of the network for previous inputs. The state data is for use by the set of computation circuits when executing a set of the nodes of the neural network for the particular input.

Circuit for executing stateful neural network

Some embodiments of the invention provide a three-dimensional (3D) circuit that is formed by vertically stacking two or more integrated circuit (IC) dies to at least partially overlap. In this arrangement, several circuit blocks defined on each die (1) overlap with other circuit blocks defined on one or more other dies, and (2) electrically connect to these other circuit blocks through connections that cross one or more bonding layers that bond one or more pairs of dies. In some embodiments, the overlapping, connected circuit block pairs include pairs of computation blocks and pairs of computation and memory blocks. The connections that cross bonding layers to electrically connect circuit blocks on different dies are referred to below as z-axis wiring or connections. This is because these connections traverse completely or mostly in the z-axis of the 3D circuit, with the x-y axes of the 3D circuit defining the planar surface of the IC die substrate or interconnect layers. These connections are also referred to as vertical connections to differentiate them from the horizontal planar connections along the interconnect layers of the IC dies.

3D processor having stacked integrated circuit die

Some embodiments provide a method for training a machine-trained (MT) network to classify inputs into multiple categories. The method propagates a set of input training items through the MT network to generate a set of corresponding outputs. Each input training item belongs to a category and the output for each input training item includes, for each category, a computed probability of the input belonging to the category. The method computes a value for a loss function based on the generated outputs. The loss function includes a first term based on the computed probabilities of each input belonging to its category and not based on individual computed probabilities of the inputs belonging to other categories and a second term based on the individual computed probabilities of each input belonging to each of the categories. The method uses the computed value for the loss function to train the MT network.

Neural network loss function that incorporates incorrect category probabilities

Some embodiments provide a method for training a machine-trained network that includes multiple parameters. The method propagates a batch of input training items through the network to generate output values and compute values of a loss function for each of the input training items. The method computes a weight for each input training item based on the computed loss function values for each of the input training items. The method selects input training items with larger weights more often than input training items with smaller weights for subsequent batches of input training items.

Weighted selection of inputs for training machine-trained network

Direct-bonded native interconnects and active base dies are provided. In a microelectronic architecture, active dies or chiplets connect to an active base die via their core-level conductors. These native interconnects provide short data paths, which forgo the overhead of standard interfaces. The system saves redistribution routing as the native interconnects couple in place. The base die may contain custom logic, allowing the attached dies to provide stock functions. The architecture can connect diverse interconnect types and chiplets from various process nodes, operating at different voltages. The base die may have state elements for drive. Functional blocks aboard the base die receive native signals from diverse chiplets, and communicate with all attached chiplets. The chiplets may share processing and memory resources of the base die. Routing blockages are minimal, improving signal quality and timing. The system can operate at dual or quad data rates. The architecture facilitates ASIC, ASSP, and FPGA ICs and neural networks, reducing footprint and power requirements.

Direct-bonded native interconnects and active base die

that is captured by one or more images (e.g., by a video or several pictures of the scene). The method analyzes a captured image to identify a unique predefined displayed content (e.g., a first pattern or a sub-pattern of the first pattern) in a first region of the image. When the predefined displayed content is identified in the captured image, the method analyzes a second region of the image that is associated with the first region, in order to determine whether a set of one or more objects is within the second region. The second region is the same as the first region in some embodiments, while in other embodiments it is a bounding region (e.g., bounding shape) that encompasses the first region and a third region around the first region. In yet other embodiments, the second region has another association with the first region. When the method determines that an object is within the second region, the method generates and sends a notification (e.g., an email, a text message, an in-app message, etc.) regarding the detection of the object in the image.

Visual wake content for image processing

Some embodiments provide an IC for implementing a machine-trained network with multiple layers. The IC includes a set of circuits to compute a dot product of (i) a first number of input values computed by other circuits of the IC and (ii) a set of predefined weight values, several of which are zero, with a weight value for each of the input values. The set of circuits includes (i) a dot product computation circuit to compute the dot product based on a second number of inputs and (ii) for each input value, at least two sets of wires for providing the input value to at least two of the dot product computation circuit inputs. The second number is less than the first number. Each input value with a corresponding weight value that is not equal to zero is provided to a different one of the dot product computation circuit inputs.

Reduced dot product computation circuit

Some embodiments provide an electronic device that includes a set of processing units and a set of machine-readable media. The set of machine-readable media stores sets of instructions for applying a network of computation nodes to an input received by the device. The set of machine-readable media stores at least two sets of machine-trained parameters for configuring the network for different types of inputs. A first of the sets of parameters is used for applying the network to a first type of input and a second of the sets of parameters is used for applying the network to a second type of input.

Training network with discrete weight values

Some embodiments of the invention provide a three-dimensional (3D) circuit that is formed by stacking two or more integrated circuit (IC) dies to at least partially overlap and to share one or more interconnect layers that distribute power, clock and/or data-bus signals. The shared interconnect layers include interconnect segments that carry power, clock and/or data-bus signals. In some embodiments, the shared interconnect layers are higher level interconnect layers (e.g., the top interconnect layer of each IC die). In some embodiments, the stacked IC dies of the 3D circuit include first and second IC dies. The first die includes a first semiconductor substrate and a first set of interconnect layers defined above the first semiconductor substrate. Similarly, the second IC die includes a second semiconductor substrate and a second set of interconnect layers defined above the second semiconductor substrate. As further described below, the first and second dies in some embodiments are placed in a face-to-face arrangement (e.g., a vertically stacked arrangement) that has the first and second set of interconnect layers facing each other. In some embodiments, a subset of one or more interconnect layers of the second set interconnect layers of the second die has interconnect wiring that carries power, clock and/or data-bus signals that are supplied to the first IC die.

3D chip sharing data bus

Some embodiments provide a method for a circuit that executes a neural network including multiple nodes. The method loads a set of weight values for a node into a set of weight value buffers, a first set of bits of each input value of a set of input values for the node into a first set of input value buffers, and a second set of bits of each of the input values into a second set of input value buffers. The method computes a first dot product of the weight values and the first set of bits of each input value and a second dot product of the weight values and the second set of bits of each input value. The method shifts the second dot product by a particular number of bits and adds the first dot product with the bit-shifted second dot product to compute a dot product for the node.

Computation of neural network node with large input values

Some embodiments provide a compiler for optimizing the implementation of a machine-trained network (e.g., a neural network) on an integrated circuit (IC). The compiler of some embodiments receives a specification of a machine-trained network including multiple layers of computation nodes and generates a graph representing options for implementing the machine-trained network in the IC. In some embodiments, the compiler also generates instructions for gating operations. Gating operations, in some embodiments, include gating at multiple levels (e.g., gating of clusters, cores, or memory units). Gating operations conserve power in some embodiments by gating signals so that they do not reach the gated element or so that they are not propagated within the gated element. In some embodiments, a clock signal is gated such that a register that transmits data on a rising (or falling) edge of a clock signal is not triggered.

Compiler for implementing gating functions for neural network configuration

Some embodiments provide a three-dimensional (3D) circuit structure that has two or more vertically stacked bonded layers with a machine-trained network on at least one bonded layer. As described above, each bonded layer can be an IC die or an IC wafer in some embodiments with different embodiments encompassing different combinations of wafers and dies for the different bonded layers. The machine-trained network in some embodiments includes several stages of machine-trained processing nodes with routing fabric that supplies the outputs of earlier stage nodes to drive the inputs of later stage nodes. In some embodiments, the machine-trained network is a neural network and the processing nodes are neurons of the neural network. In some embodiments, one or more parameters associated with each processing node (e.g., each neuron) is defined through machine-trained processes that define the values of these parameters in order to allow the machine-trained network (e.g., neural network) to perform particular operations (e.g., face recognition, voice recognition, etc.). For example, in some embodiments, the machine-trained parameters are weight values that are used to aggregate (e.g., to sum) several output values of several earlier stage processing nodes to produce an input value for a later stage processing node.

Three dimensional circuit implementing machine trained network

Some embodiments provide a method for training a machine-trained (MT) network that processes inputs using network parameters. The method propagates a set of input training items through the MT network to generate a set of output values. The set of input training items comprises multiple training items for each of multiple categories. The method identifies multiple training item groupings in the set of input training items. Each grouping includes at least two training items in a first category and at least one training item in a second category. The method calculates a value of a loss function as a summation of individual loss functions for each of the identified training item groupings. The individual loss function for each particular training item grouping is based on the output values for the training items of the grouping. The method trains the network parameters using the calculated loss function value.

Using batches of training items for training a network

Some embodiments provide a method that receives a specification of a neural network for execution by an integrated circuit. The integrated circuit includes a neural network inference circuit for executing the neural network to generate an output based on an input, an input processing circuit for providing the input to the neural network inference circuit, a microprocessor circuit for controlling the neural network inference circuit and the input processing circuit, and a unified memory accessible by the microprocessor circuit, the neural network inference circuit, and the input processing circuit. The method determines usage of the unified memory by the neural network inference circuit while executing the neural network. Based on the determined usage by the neural network inference circuit, the method allocates portions of the unified memory to the microprocessor circuit and input processing circuit.

Allocating blocks of unified memory for integrated circuit executing neural network

Stacked IC structure with orthogonal interconnect layers

Growing community of inventors

Menlo Park, CA, United States of America

Steven L Teig