Sima Technologies, Inc.

Microsoft Technology Licensing, LLC

Nvidia Corporation

Intel Corporation

Nishit Harshad Shah

Reed Kotler

Srivathsa Dhruvanarayan

Tezaswi Raja

Ankit More

Mattheus C Heddes

Moenes Zaher Iskarous

Kavitha Prasad

Spenser Don Gilliland

Yogesh Laxmikant Chobe

Sedny S J Attia

Torsten Hoefler

Gunaseelan Ponnuvel

Ting Ku

Suhas Satheesh

Krishnamraju Kurra

Suhas Mysore Satheesh

Miguel Rodriguez

Bradley L Taylor

Ravishankar Rao

Circuitry and a method of determining electrical characteristics of material local to a specific area of a semiconductor wafer. In one embodiment, the method includes sinking or sourcing current through a selected on of a plurality of devices under test (DUTs) on the semiconductor wafer, converting the current sourcing or sinking into a voltage, comparing the converted voltage against a linear voltage ramp, generating an output clock based on the comparison, and measuring a duty cycle of the output clock. In one embodiment, the duty cycle of the output clock is dependent on the current sinking or sourcing through the selected at least one of the plurality of DUTs on the wafer and electrical characteristics of material local to the specific area of the wafer where the selected one of the plurality of DUTs is located are determined based on the duty cycle of the output clock.

Integrated current monitor

Circuitry and a method of determining electrical characteristics of material local to a specific area of a semiconductor wafer is disclosed. In one embodiment, an IC is disclosed that comprises at least one processing subsystem and at least one integrated common current monitor (ICCM) located within the processing subsystem(s). In one embodiment, the ICCM includes current-to-voltage conversion circuitry that converts a current throughput (I) of a selected at least one of the plurality of DUTs to a corresponding voltage for a plurality of regulated drain-to-source voltages (V) across the selected DUT(s). In one embodiment, the ICCM is configured to determine a duty cycle of a voltage that corresponds to the Iand Irepresents electrical characteristics of material local to an area of a semiconductor wafer specific to a location where the ICCM is located.

Integrated current monitor using variable drain-to-source voltages

A method, system, and apparatus are disclosed herein for bridging a deterministic phase of instructions with a non-deterministic phase of instructions when those instructions are executed by a machine learning accelerator while executing a machine learning network. In the non-deterministic phase, data and instructions are transferred from off-chip memory to on-chip memory. When the transfer is complete, processing elements are synchronized and, upon synchronization, a deterministic phase of instructions is executed by the processing elements.

Synchronization of processing elements that execute statically scheduled instructions in a machine learning accelerator

Embodiments of the present disclosure include a digital circuit and method for multi-stage compression. Digital data values are compressed using a multi-stage compression algorithm and stored in a memory. A decompression circuit receives the values and performs a partial decompression. The partially compressed values are provided to a processor, which performs the final decompression. In one embodiment, a vector of N length compressed values are decompressed using a first bit mask into two N length sets having non-zero values. The two N length sets are further decompressed using two M length bit masks into M length sparse vectors, each having non-zero values.

Method and apparatus for compressing and decompressing sparse data sets

A method, system, and apparatus are disclosed herein for bridging a deterministic phase of instructions with a non-deterministic phase of instructions when those instructions are executed by a machine learning accelerator while executing a machine learning network. Specifically, data is transferred from off-chip memory to on-chip memory (non-deterministic phase of instructions). The data transfer involves determining whether certain on-chip memory is already storing data that has not been consumed yet (e.g., certain memory locations on-chip may be storing data for future consumption and should not be overwritten). Based on determining that the certain on-chip memory is not storing data that has not been consumed yet, the data may be transferred from the off-chip memory to the on-chip memory and the target memory locations may be marked as storing data that has not been consumed yet. The deterministic phase of instructions may be started subsequently.

Software managed memory hierarchy

Embodiments of the present disclosure include techniques for compressing data using a tree encoded bit mask that may result in higher compression ratios. In one embodiment, an input vector having a plurality of values is received by a first plurality of switch circuits. Selection of the input values is controlled by sets of bits from the bit mask. The sets of bits specify locations of portions of the input vector where particular value of interest reside. The switch circuits output multiple values of the input vector, which include the particular value of interest. A second stage of switch circuits is controlled by logic circuit that detects values on the outputs of the first stage of switch circuits and outputs the values of interest. In some embodiments, the values of interest may be non-zero values of a sparse input vector, and the switch circuits may be multiplexers.

Compression circuits and methods using tree based encoding of bit masks

A compiler generates a computer program implementing a machine learning network on a machine learning accelerator (MLA) including interconnected processing elements. The computer program includes data transfer instructions for non-colliding data transfers between the processing elements. To generate the data transfer instructions, the compiler determines non-conflicting data transfer paths for data transfers based on a topology of the interconnections between processing elements, on dependencies of the instructions and on a duration for execution of the instructions. Each data transfer path specifies a routing and a time slot for the data transfer. The compiler generates data transfer instructions that specify routing of the data transfers and generates a static schedule that schedules execution of the data transfer instructions during the time slots for the data transfers. The static schedule also schedules execution of compute instructions for computations using transferred data that implement the machine learning network.

Inter-processor data transfer in a machine learning accelerator, using statically scheduled instructions

Embodiments of the present disclosure include a digital circuit and method for compressing input digital values. A plurality of input digital values may include zero values and non-zero values. The input digital values are received on M inputs of a first switching stage. The first switching stage is arranged in groups that rearrange the non-zero values on first switching stage outputs according to a compression and shift. The compression and shift position the non-zero values on outputs coupled to inputs of a second switching stage. The second switching stage consecutively couples non-zero values to N outputs, where N is less than M.

Method and apparatus for compression multiplexing for sparse computations

A compiler manages memory usage in the machine learning accelerator by intelligently ordering computations of a machine learning network. The compiler identifies partial networks of the machine learning network representing portions of the machine learning network across multiple layers on which an output or set of outputs are dependent. Because any given output may depend on only a limited subset of intermediate outputs from the prior layers, each partial network may include only a small fraction of the intermediate outputs from each layer. Instead of implementing the MLN by computing one layer at a time, the compiler schedules instructions to sequentially implement partial networks. As each layer of a partial network is completed, the intermediate outputs can be released from memory. The described technique enables intermediate outputs to be directly streamed between processing elements of the machine learning accelerator without requiring large transfers to and from external memory.

Ordering computations of a machine learning network in a machine learning accelerator for efficient memory usage

In various embodiments, a comparison circuit compares voltages within an integrated circuit. The comparison circuit includes a comparison capacitor, an inverter, and multiple switches. A first terminal of the comparison capacitor is coupled to both a first terminal of a first switch and a first terminal of a second switch. A second terminal of the comparison capacitor is coupled to both a first terminal of a third switch and an input of the inverter. An output of the inverter is coupled to both a second terminal of the third switch and a first terminal of a fourth switch. A second terminal of the fourth switch is coupled to a first terminal of a fifth switch and a first output of the comparison circuit. At least a portion of the switches are turned on during a comparison model and are turned off during a reset mode.

On-die techniques for asynchnorously comparing voltages

A compiler receives a description of a machine learning network and generates a computer program that implements the machine learning network. The compiler allocates instructions of the computer program to different groups of processing elements (Tiles) for execution such that different groups of Tiles implement different layers of the machine learning network. The compiler may determine the size of the different groups based on a partial computation metric associated with the computations performed to implement the corresponding layer. Furthermore, the compiler may assign specific Tiles to each group based on a set of predefined layout constraints. The compiler may statically schedule at least a portion of the instructions into one or more deterministic phases for execution by the groups of Tiles.

Allocating computations of a machine learning network in a machine learning accelerator

A compiler receives a description of a machine learning network (MLN) and generates a computer program that implements the MLN on a machine learning accelerator (MLA). To implement the MLN, the compiler generates compute instructions that implement computations of the MLN on different processing units (Tiles), and data transfer instructions that transfer data used in the computations. The compiler may statically schedule at least a portion of the instructions for execution by the Tiles according to fixed timing. The compiler may initially implement data transfers between non-adjacent Tiles (or external memories) by implementing a sequence of transfers through one or more intermediate Tiles (or external memories) in accordance with a set of default routing rules that dictates the data path. The computer program may then be simulated to identify routing conflicts. When routing conflicts are detected, the compiler updates the computer program in a manner that avoids the conflicts.

Avoiding data routing conflicts in a machine learning accelerator

A system-on-chip (SoC) integrated circuit product includes a machine learning accelerator (MLA). It also includes other processor cores, such as general purpose processors and application-specific processors. It also includes a network-on-chip for communication between the different modules. The SoC implements a heterogeneous compute environment because the processor cores are customized for different purposes and typically will use different instruction sets. Applications may use some or all of the functionalities offered by the processor cores, and the processor cores may be programmed into different pipelines to perform different tasks.

Heterogeneous computing on a system-on-chip, including machine learning inference

In various embodiments, a current measurement circuit measures an input current within an integrated circuit. The current measurement circuit includes an integration capacitor, an operational amplifier, a comparison capacitor, an inverter, and multiple switches. The current measurement circuit is coupled to a clocking circuit that, during operation, generates a two-phase clock having a frequency that is proportional to the input current. At least a portion of the switches are turned on during a first phase of the two-phase clock and are turned off during a second phase of the two-phase clock.

Growing community of inventors

Sunnyvale, CA, United States of America

Nishit Harshad Shah