Black Sesame Technologies Inc.

Zheng Qi

Yi Wang

Zheng Li

Qun Gu

Ying Zhou

Chengyu Xiong

Jizhang Shan

Yu Huang

Hui Wang

Zuoguan Wang

Qiang Zhang

Ruihui Di

Tian Zhou

Wei Zuo

Chenhao Fang

Chenghao Zhang

The present invention discloses a graph partitioning system for running neural networks on resource constrained hardware systems. The graph partitioning system used for partitioning a neural network graph into a series of sub-graphs and further allow the multiple sub-graphs to be executed in available hardware subsystems. The system based on cost function as estimated computation time and memory bandwidth of partitioned sub-graphs. The graph partitioning system is a cycle estimation model of hardware that can run fast and parameterize memory latency. The graph partitioning system supports heterogeneous partition for different type accelerators such as CPU, GPU, ASIC. The present invention also discloses a method for partitioning neural network graph in to series of sub-graphs.

Neural networks graph partitioning system and method for the same

The present invention relates to a method and a system for performing depthwise separable convolution on an input data in a convolutional neural network. The invention utilizes a heterogeneous architecture with a number of MAC arrays including 1D MAC arrays and 2D MAC arrays with a Winograd conversion logic to perform depthwise separable convolution. The depthwise separable convolution uses less weight parameters and thus less multiplications while it obtains the same computation results as the traditional convolution.

Heterogeneous architecture for depthwise-seperable convolution based neural network computation acceleration

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for controlling, by an on-chip memory controller, a plurality of hardware components that are configured to perform computations to access a shared memory. One of the on-chip memory controller includes at least one backside arbitration controller communicatively coupled with a memory bank group and a first hardware component, wherein the at least one backside arbitration controller is configured to perform bus arbitrations to determine whether the first hardware component can access the memory bank group using a first memory access protocol; and a frontside arbitration controller communicatively coupled with the memory bank group and a second hardware component, wherein the frontside arbitration controller is configured to perform bus arbitrations to determine whether the second hardware component can access the memory bank group using a second memory access protocol different from the first memory access protocol.

High-performance on-chip memory controller

An example fused convolutional layer, comprising, a comparator capable of reception of a first zero point and a multiply-accumulation result, a first multiplexer coupled to the comparator, wherein the first multiplexer receives a plurality of power-of-two exponent values, a shift normalizer, coupled to the first multiplexer, wherein the shift normalizer is capable of receiving the multiply-accumulation result and the plurality of power-of-two exponent values, wherein the shift normalizer limits a quantization of the multiply-accumulation result to a power-of-two scale and a second multiplexer coupled to an output of the shift normalizer, the first multiplexer and receives a second zero point and outputs an activation.

Integer-based fused convolutional layer in a convolutional neural network

An extensible multi-precision data pipeline system, comprising, a local buffer that stores an input local data set in a local storage format, an input tensor shaper coupled to the local buffer that reads the input local data set and converts the input local data set into an input tensor data set having a tensor format of vector width N by tensor length L, a cascaded pipeline coupled to the input tensor shaper that routes the input tensor data set through at least one function stage resulting in an output tensor data set, an output tensor shaper coupled to the cascaded pipeline that converts the output tensor data set into an output local data set having the local storage format and wherein the output tensor shaper writes the output local data set to the local buffer.

Extensible multi-precision data pipeline for computing non-linear and arithmetic functions in artificial neural networks

A system on a chip, including a first domain having a first processor, a first local memory coupled to the first processor, wherein the first local memory having a first memory format and a first sub-network coupled to the first processor, a second domain having a second processor, a second local memory coupled to the second processor and a second sub-network coupled to the second processor, wherein the second local memory having a second memory format which differs from the first memory format, a multi-tier network coupled to the first sub-network and the second sub-network, a global memory coupled to the multi-tier network and a multi-port DDR controller coupled to the global memory to receive, transmit and share the first local memory having the first memory format and the second local memory having the second memory format based on a predetermined criteria.

Heterogeneous computation and hierarchical memory image sensing pipeline

The invention discloses a method and a system for achieving a unified flow control system for multiple camera devices. The system provides inline and offline streams to share resources by converting the streams into one another. The resource sharing is performed by using different time interval to process inline and offline streams. The system also includes a STALL & REDO operation to keep whole image un-broken and shut down the write stream from ISP right away.

Unified flow control for multi-camera system

A method of wheel encoder to camera calibration, including receiving a LiDAR (Light Detection and Ranging) signal, receiving a camera signal, receiving a wheel encoder signal, calibrating the camera signal to the LiDAR signal, calibrating the wheel encoder signal to the LiDAR signal and calibrating the camera signal to the wheel encoder signal based on the calibration of the camera signal to the LiDAR signal and the wheel encoder signal to the LiDAR signal.

LiDAR assisted wheel encoder to camera calibration

An example method of image signal processing, comprising at least one of, receiving a set of high priority signals, receiving a set of low priority signals, reconfiguring a first portion of a pipeline to route the high priority signals through an in-line mode process and reconfiguring a second portion of the pipeline to route the low priority signals through an offline mode process.

Growing community of inventors

Cupertino, CA, United States of America

Zheng Qi