Marvell Asia Pte., Ltd.

Intel Corporation

Avinash Sodani

Chia-Hsin Chen

Ulf Hanebutte

Senad Durakovic

Stephan Jourdan

Per H Hammarlund

Alexandre J Farcy

Ramacharan Sundararaman

Hamid Reza Ghasemi

Srinivas Sripada

Gopal Nalamalapu

Matthew C Merten

James Hadley

Glenn J Hinton

Sebastien Hily

Atul Bhattarai

Ronak Singhal

Morris Marden

Robert J Kyanko

Milind Baburao Girkar

Deborah T Marr

Richard J Greco

Andreas Kleen

John G Holm

Nikhil Jayakumar

Ilhyun Kim

Krishna N Vinod

Steven E Raasch

Christopher M Cantalupo

Richard David Taylor

Mark C Davis

Benjamin C Chaffin

Vijaykumar Balaram Kadgi

Samie B Samaan

James G Eldredge

Nithyananda Miyar

Dan Tu

Iredamola Dammy Olopade

Rishan Tan

Michael Dudek

Darren Walworth

Roshan Fernando

James Irvine

Mani Gopal

Zainulabedin J Aurangabadwala

Zeev Sperber

Shubhendu Sekhar Mukherjee

Rajat Agarwal

Enrique Musoll

Michael A Fetterman

Robert S Chappell

Sandeep Ahuja

Michael Cornaby

Aditya Kesiraju

Adrian C Moga

Roger Gramunt

Arijit Biswas

Saurabh Shrivastava

Enric Musoll

Ramon Matas

Laura A Knauth

Julio Gago

Ashok Jagannathan

Alexey Suprun

Chien-Chun Chou

Meenakshisundaram Chinthamani

Harri Hakkarainen

Chung-Lun Chan

Timothy Y Kam

Michael Fetterman

Rammohan Padmanabhan

Martin Kovac

Neal S Moyer

Prabhat Jain

Soumya Gollamudi

David K Li

Raghuveer Shivaraj

Rahul R Kulkarni

Joseph A McMahon

Jinho Suh

Vikram S Sundaram

Gerardo A Fernandez

Michael S Yang

Grace C Lee

Fariborz Tabesh

A new approach is proposed to support SRAM less bootup of an electronic device. A portion of a cache unit of a processor is utilized as a SRAM to maintain data to be accessed via read and/or write operations for bootup of the electronic device. First, the portion of the cache unit is mapped to a region of a memory, which has not been initialized. The processor reads data from a non-modifiable storage to be used for the bootup process of the electronic device and writes the data into the portion of the cache unit serving as the SRAM. To prevent having to read or write to the uninitialized memory, any read operation to the memory region returns a specific value and any write operation to the memory region is dropped. The processor then accesses the data stored in the portion of the cache unit to bootup the electronic device.

System and method for SRAM less electronic device bootup using cache

A programmable hardware system for machine learning (ML) operations includes a core and an inference engine. The core receives commands from a host. The commands are in a first instruction set architecture (ISA) format. The core divides the commands into a first set for performance-critical operations, in the first ISA format, and a second set of performance non-critical operations, in the first ISA format. The core executes the second set to perform the performance non-critical operations of the ML operations and streams the first set to inference engine. The inference engine generates a stream of the first set of commands in a second ISA format based on the first set of commands in the first ISA format. The first set of commands in the second ISA format programs components within the inference engine to execute the ML operations to infer data.

Instruction set architecture (ISA) format for multiple instruction set architectures in machine learning inference engine

A method includes receiving a set of data. The set of data is divided into a plurality of data portions. The method includes transmitting the plurality of data portions to a plurality of processing tiles, wherein each data portion of the plurality of data portions is associated with a processing tile of a plurality of tiles. Each processing tile of the plurality of tiles performs at least one local operation on its respective data portion to form a local result. The method includes exchanging local results between the plurality of processing tiles. Moreover, the method includes calculating a global value based on the local results. The method further includes performing at least one local operation by each processing tile of the plurality of tiles on its respective data portion based on the global value to form a computed result.

Method and apparatus for performing machine learning operations in parallel on machine learning hardware

A programmable hardware system for machine learning (ML) includes a core and a streaming engine. The core receives a plurality of commands and a plurality of data from a host to be analyzed and inferred via machine learning. The core transmits a first subset of commands of the plurality of commands that is performance-critical operations and associated data thereof of the plurality of data for efficient processing thereof. The first subset of commands and the associated data are passed through via a function call. The streaming engine is coupled to the core and receives the first subset of commands and the associated data from the core. The streaming engine streams a second subset of commands of the first subset of commands and its associated data to an inference engine by executing a single instruction.

Streaming engine for machine learning architecture

A system to support a machine learning (ML) operation comprises an array-based inference engine comprising a plurality of processing tiles each comprising at least one or more of an on-chip memory (OCM) configured to maintain data for local access by components in the processing tile and one or more processing units configured to perform one or more computation tasks on the data in the OCM by executing a set of task instructions. The system also comprises a data streaming engine configured to stream data between a memory and the OCMs and an instruction streaming engine configured to distribute said set of task instructions to the corresponding processing tiles to control their operations and to synchronize said set of task instructions to be executed by each processing tile, respectively, to wait current certain task at each processing tile to finish before starting a new one.

Architecture to support color scheme-based synchronization for machine learning

A processing unit to support inference acceleration for machine learning (ML) comprises an inline post processing unit configured to accept and maintain one or more lookup tables for performing a tanh and/or sigmoid operation/function. The inline post processing unit is further configured to accept data from a set of registers configured to maintain output from a processing block instead of streaming the data from an on-chip memory (OCM), perform the tanh and/or sigmoid operation on each element of the data from the processing block on a per-element basis via the one or more lookup tables, and stream post processing result of the per-element tanh and/or sigmoid operation back to the OCM after the tanh and/or sigmoid operation is complete.

Architecture to support tanh and sigmoid operations for inference acceleration in machine learning

A method includes receiving a first set of data. The method also includes receiving an instruction to determine a largest value within the first set of data. The first set of data is divided into a first plurality of data portions based on a hardware architecture of a first plurality of processing elements. The first plurality of data portions is mapped to the first plurality of processing elements. Each data portion of the first plurality of data portions is mapped exclusively to a processing element of the first plurality of processing elements. Each data portion of the first plurality of data portions is processed by its respective processing element to identify a largest value from each data portion of the first plurality of data portions, wherein the processing forms a first output data comprising the largest value from the each data portion of the first plurality of data portions.

A system includes a first and a second group of cores in a multicore system. Each core of the first/second group is configured to process data. Each core within the first/second group is configured to enter into an idle state in response to being idle for a first/second period of time respectively. Every idle core in the first/second group is configured to transition out of the idle state and into an operational mode in response to receiving a signal having a first/second value respectively and further in response to having a pending operation to process.

Power management and staggering transitioning from idle mode to operational mode

A method of converting a data stored in a memory from a first format to a second format is disclosed. The method includes extending a number of bits in the data stored in a double data rate (DDR) memory by one bit to form an extended data. The method further includes determining whether the data stored in the DDR is signed or unsigned data. Moreover, responsive to determining that the data is signed, a sign value is added to the most significant bit of the extended data and the data is copied to lower order bits of the extended data. Responsive to determining that the data is unsigned, the data is copied to lower order bits of the extended data and the most significant bit is set to an unsigned value, e.g., zero. The extended data is stored in an on-chip memory (OCM) of a processing tile of a machine learning computer array.

System and method for INT9 quantization

A system includes a power profile engine, a power measurement engine, and a power throttling signal generator. The power profile engine receives a desired power profile, e.g., a first profile current average associated with a first time duration and a second profile current average associated with a second time duration. The power measurement engine measures current being drawn and generates a first running average for the measured currents for the first time duration and generates a second running average for the measured currents for the second time duration. The power throttling signal generator generates a first power throttling signal to throttle power in response to the first running average for the measured currents being greater than the first profile current average and generates a second power throttling signal to throttle power in response to the second running average for the measured currents being greater than the second profile current average.

System and method to manage power to a desired power profile

A new approach is proposed that contemplates systems and methods to support post reset fuse reload for latency reduction. First, values of fuses are read once and stored into one or more load registers on an electronic device, wherein the load registers are protected. Once the values of the fuse are loaded into the load registers, a valid indicator of the load registers is set indicating that the values have been successfully loaded into the load registers. When other components of the electronic device need to access these values, the other components will check the load registers first. If it is determined that the valid indicator of the load registers is set, the stored values are read from the load registers instead of from the fuses. If the valid indicator of the load registers is not set, the values are loaded again from the fuses into the load registers.

System and methods for latency reduction for fuse reload post reset

A new approach for supporting tag-based synchronization among different tasks of a machine learning (ML) operation. When a first task tagged with a set tag indicating that one or more subsequent tasks need to be synchronized with it is received at an instruction streaming engine, the engine saves the set tag in a tag table and transmits instructions of the first task to a set of processing tiles for execution. When a second task having an instruction sync tag indicating that it needs to be synchronized with one or more prior tasks is received at the engine, the engine matches the instruction sync tag with the set tags in the tag table to identify prior tasks that the second task depends on. The engine holds instructions of the second task until these matching prior tasks have been completed and then releases the instructions to the processing tiles for execution.

System and methods for tag-based synchronization of tasks for machine learning operations

A new approach is proposed to support hardware-based protection for registers of an electronic device. Sources requesting access to the registers are categorized into a set of internal sources that can be trusted and a set of external sources that are untrusted. The registers are classified into a set of internal registers allowed to be accessed by the internal resources only, a set of read-only external registers that can be read by the external resources in addition to accessed by the internal resources, and a set of read/write external registers that can be read and written by both the internal and the external resources. Each access request by a source to the registers includes the source type, wherein access request is granted or denied based on the matching between the source bits in the access request and the register classification bits of the one or more registers to be accessed.

System and method for hardware-based register protection mechanism

A system includes a multicore chip configured to perform machine learning (ML) operations. The system also includes a power monitoring module configured to measure power consumption of the multicore chip on a main power rail of the multicore chip. The power monitoring module is further configured to assert a signal in response to the measured power consumption exceeding a first threshold. The power monitoring module is further configured to transmit the asserted signal to a power throttling module to initiate a power throttling for the multicore chip.

Power management and current/ramp detection mechanism

A system includes a memory, an interface engine, and a master. The memory is configured to store data. The inference engine is configured to receive the data and to perform one or more computation tasks of a machine learning (ML) operation associated with the data. The master is configured to interleave an address associated with memory access transaction for accessing the memory. The master is further configured to provide a content associated with the accessing to the inference engine.

Address interleaving for machine learning

Growing community of inventors

San Jose, CA, United States of America

Avinash Sodani