Amazon Technologies, Inc.

Iii Holdings 2, LLC

Motorola Corporation

Analog Devices,inc.

Other

Thomas A Volpe

Mark Bradley Davis

Ron Diamant

Mark Banse

Nafea Bshara

Kari Ann O'Brien

Kiran Kalkunte Seshadri

Bijendra Singh

Steven Scott Larson

Thomas Elmer

David James Borland

Sundeep Amirineni

Raymond Scott Whiteside

Randy Renfu Huang

Robert Michael Johnson

Ken Goss

Asif Khan

Danny Marquette

Jason Hobbs

Nishith Desai

Adi Habusha

Dana Michelle Vantrease

Kun Xu

James G Gay

Yaniv Shapira

Declan Conlon

Nancy G Woodbridge

Joshua Wayne Bowman

Marc John Brooker

Norbert Paul Kusters

Prashant R Chandra

Marc Stephen Olson

Paul Gilbert Meyer

Leonard Thomas Tracy

Michael S Allen

Brian Robert Silver

Timothy David Gasser

Kenneth S Goss

Niall Joseph Dalton

Asif Kahn

Carlos Javier Cabral

Michael R Miller

Vithal Dattatraya Shirodkar

Kari Ann OBrien

Thomas Bradley Scholl

Alan Michael Judge

Jindrich Zejda

Justin Oliver Pietsch

Drazen Borkovic

Jamie Plenderleith

Mark Noel Kelly

Alaa Adel Mahdi Hayder

Stephen Callaghan

Alwood Patrick Williams, Iii

Gianluca Grilli

Edward Crabbe

Bassam Abdel-Dayem

Todd Swanson

Fnu Arun Kumar

Aaron Bauch

Douglas Lloyd Mainz

Vasanta Kumar Palisetti

Chin Cheah

Alwood Patrick Williams

Systems and methods are provided to perform multiply-accumulate operations of reduced precision numbers in a systolic array. Each row of the systolic array can receive reduced inputs from a respective reducer. The reducer can receive a particular input and generate multiple reduced inputs from the input. The reduced inputs can include reduced input data elements and/or a reduced weights. The systolic array may lack support for inputs with a first bit-length and the reducers may reduce the bit-length of a given input from the first bit-length to a second shorter bit-length and provide multiple reduced inputs with second shorter bit-length to the array. The systolic array may perform multiply-accumulate operations on each unique combination of the multiple reduced input data elements and the reduced weights to generate multiple partial outputs. The systolic array may sum the partial outputs to generate the output.

Systolic array with input reduction to multiple reduced inputs

To reduce power consumption, data bits or a portion of a data register that is not expected to toggle frequently can be grouped together, and be clock-gated independently from the rest of the data register. The grouping of the data bits can be determined based on the data types of the workload being operated on. For a data register configured to store a numeric value that supports multiple data types, the portion of the data register being clock-gated may store a group of data bits that are unused for one or more data types of the multiple data types supported by the data register. The portion of the data register being clock-gated can also be a group of data bits that remain unchanged or have a constant value for numeric values within a certain numeric range that is frequently operated on.

Data-type-aware clock-gating

On-circuit activity monitoring may be performed to modify integrated circuit processing. An activity monitor may be implemented on an integrated circuit to monitor activity measurements of processing data at another portion of the integrated circuit. A change to activity measurements may be detected and cause the activity monitor to modify the rate at which data enters the other portion of the integrated circuit for processing.

On-circuit activity monitoring for modifying integrated circuit processing

On-circuit utilization monitoring may be performed for a systolic array. A current utilization measurement may be determined for processing elements of a systolic array and compared with a prior utilization measurement. Based on the comparison, a throttling recommendation may be provided to a management component to determine whether to perform the throttling recommendation.

On-circuit utilization monitoring for a systolic array

Systems and methods are provided to enable parallelized multiply-accumulate operations in a systolic array. Each column of the systolic array can include multiple busses enabling independent transmission of input partial sums along the respective bus. Each processing element of a given columnar bus can receive an input partial sum from a prior element of the given columnar bus, and perform arithmetic operations on the input partial sum. Each processing element can generate an output partial sum based on the arithmetic operations, provide the output partial sum to a next processing element of the given columnar bus, without the output partial sum being processed by a processing element of the column located between the two processing elements that uses a different columnar bus. Use of columnar busses can enable parallelization to increase speed or enable increased latency at individual processing elements.

Multiple accumulate busses in a systolic array

In one embodiment of the disclosure, packets are routed according to a VRF (Virtual Routing and Forwarding) domain that is represented by a VRF identifier in a VRF-identifier (ID) field of packet headers. In one embodiment of the disclosure, a VRF identifier may be added to a VRF-ID field of a packet header so that the packet is routed according to a VRF domain that the packet belongs to.

Extending virtual routing and forwarding

A semiconductor package can include a capacitance die. The package can have multiple dice (e.g., logic die, memory die) mounted on a substrate. Each die can include a power domain. The dice can be distributed on the substrate such that an extra space is present on the substrate between at least some of the dice. For example, an extra space may be present between two dice, at a corner of the substrate, or other locations. The extra space can disrupt a coplanarity of the semiconductor package. The capacitance die can be located in the extra space so as to establish the coplanarity with the other dice. The capacitance die can include a capacitor array electrically coupled to multiple power domains of the plurality of dice.

Semiconductor package with capacitance die

Disclosed herein are techniques for performing multi-layer neural network processing for multiple contexts. In one embodiment, a computing engine is set in a first configuration to implement a second layer of a neural network and to process first data related to a first context to generate first context second layer output. The computing engine can be switched from the first configuration to a second configuration to implement a first layer of the neural network. The computing engine can be used to process second data related to a second context to generate second context first layer output. The computing engine can be set to a third configuration to implement a third layer of the neural network to process the first context second layer output and the second context first layer output to generate a first processing result of the first context and a second processing result of the second context.

Processing for multiple input data sets in a multi-layer neural network

In various implementations, provided are systems and methods for operating a neural network that includes conditional structures. In some implementations, an integrated circuit can compute a result using a set of intermediate results, where the intermediate results are computed from the outputs of a hidden layer of the neural network. The integrated circuit can further test the result against a condition. The outcome of the test can determine a next layer that the integrated circuit is to execute, or can be used to determine that further execution of the neural network can be terminated.

Processor with control flow

Techniques to reduce the latency in notifying that space in a memory has been freed up are described. For example, when moving data from on-chip memory of a computing engine to system memory, the computing engine can be notified that its on-chip memory is free before an acknowledgment is provided by the system memory that the data being moved has been written into the system memory. The computing engine can be given access to the on-chip memory sooner by generating an early semaphore update based on a determination that the set of data being moved to system memory has been read out from the on-chip memory. The early semaphore update need not wait for the acknowledgement from the system memory, thus reducing the latency of notifying the computing engine that the on-chip memory is free.

Early semaphore update

On-chip software-based activity monitoring is implemented to configure hardware-based activity throttling. A software-based activity monitor implemented on an integrated circuit obtains data from on-chip components to determine throttling modifications for a processing engine of the integrated circuit. The throttling modifications are applied to throttling criteria that is used by a hardware-based activity monitor on the integrated circuit which is responsible for directly evaluating and throttling processing at the processing engine of the integrated circuit.

On-chip software-based activity monitor to configure throttling at a hardware-based activity monitor

Simulated network packets may be processed via a network processing pipeline. A packet processor may implement packet processing stages to process network packets received via physical network interface at the packet processor. A controller for the packet processor may provide simulated network packets to the packet processor for processing at the different packet processing stages, bypassing the physical network interface. Dummy simulated packets may be provided to or generated at the packet processor on behalf of the controller to be parsed, processed, and returned to the controller. Metadata simulated packets may be injected into the packet processing stages and recaptured in storage locations that are accessible to the controller.

Simulating network packets in a packet processing pipeline

Systems and methods are provided to perform multiply-accumulate operations of reduced precision numbers in a systolic array. Each row of the systolic array can receive reduced inputs from a respective reducer. The reduced input can include a reduced input data element and/or a reduced weight. The systolic array may lack support for inputs with a first bit-length and the reducers may reduce the bit-length of a given input from the first bit-length to a second shorter bit-length and provide the reduced input to the array. In order to reduce the bit-length, the reducer may reduce the number of trailing bits of the input. Further, the systolic array can receive a reduced and rounded input. The systolic array can propagate the reduced input through the processing elements in the systolic array. Each processing element may include a multiplier and/or an adder to perform arithmetical operations based on the reduced input.

Systolic array with efficient input reduction and extended array performance

Disclosed are techniques for implementing features within a network device. The network device can function to forward sequences of data packets received by the network device as well as concurrently generate or check test type of data packets.

Testing computer networks in real time

Systems and methods are provided to perform multiply-accumulate operations of multiple data types in a systolic array. One or more processing elements in the systolic array can include a shared multiplier and one or more adders. The shared multiplier can include a separate and/or a shared circuitry where the shared circuitry can perform at least a part of integer multiplication and at least a part of non-integer multiplication. The one or more adders can include one or more shared adders or one or more separate adders. The shared adder can include a separate and/or a shared circuitry where the shared circuitry can perform at least a part of integer addition and at least a part of non-integer addition.

Systolic array component combining multiple integer and floating-point data types

Growing community of inventors

Austin, TX, United States of America

Thomas A Volpe