Ambarella, Inc.

Ambarella International LP

Sun Microsystems, Inc.

Lsi Logic Corporation

Oracle America, Inc.

Ambarella Taiwan Ltd.

Ambrella, Inc.

Leslie D Kohn

Elliot N Linzer

Robert C Kunz

Peter Verplaetse

Yu Wang

Michael K Wong

Ellen M Lee

Kathirgamar Aingaran

Chi Hong John Ju

Jenn-Yuan Tsai

Didier LeGall

Guy Rapaport

Sydney D Reader

Wilson Kwok

Kamil Metin Uz

Tihao Chiang

Jeffery R Campbell

Shih-Hao Wang

Kunle A Olukotun

Shih-Hsuan Tao

Gregory Frederick Grohoski

Paul Joseph Jordan

Timothy J Van Hook

David Barr

Robert Yung

Manish K Singh

Wei Fang

Didier Le Gall

Diego P De Garrido

Benghan Lui

KunIe A Olukotun

An apparatus having a processor and a circuit is disclosed. The processor may be configured to (i) compare, at a first level of a motion estimation hierarchy, first units of a current picture with a reference picture to generate first metrics, (ii) combine, at the first level, the first metrics to generate second metrics and (iii) refine, at a second level of the hierarchy, the first metrics and the second metrics to generate motion vectors. Multiple metrics may be refined in parallel. The first metrics generally correspond to the first units in an overlapping unit of the current picture. The second metrics generally correspond to a plurality of second units in the overlapping unit. Each second unit may overlap one or more first units. The circuit may be configured to process the overlapping unit based on the motion vectors to generate an output signal.

Shared motion estimation cost metrics for overlapping units

An apparatus includes a memory and a hardware pipeline. The memory may be configured to store video data. The video data includes a plurality of sections of one or more pictures that may be processed independently. The hardware pipeline comprises a plurality of pipeline stages implementing a video coding process comprising a number of steps. Each of the plurality of pipeline stages performs an associated task of a different step of the video coding process in a substantially similar time on a different one of the plurality of sections as each of the plurality of sections pass through each of the pipeline stages. At least one of the plurality of pipeline stages communicates predictor information that is based on actual neighbor data to an earlier stage of the hardware pipeline.

Interleaved video coding pipeline

An apparatus includes a first circuit and a second circuit. The first circuit may be configured to calculate a feature descriptor for each pixel of an image. The feature descriptor generally comprises sixteen discriminative feature values. The feature values are generally computed based on arithmetic combinations of a number of neighboring pixels at predetermined locations around each pixel. The second circuit may be configured to implement a color classification process combining multiple weak depth-two decision trees in a cascade. The second circuit may generate a plurality of likelihood values expressing a likelihood of a specific color road marking at each pixel location of the image using the feature descriptor of each pixel of the image.

Real-time color classification for street vehicles

An apparatus includes a scheduler circuit and a processing circuit. The scheduler circuit may be configured to (i) parse a directed acyclic graph into one or more operators and (ii) schedule the one or more operators in one or more data paths. The processing circuit generally comprises one or more hardware engines configured as the one or more data paths. The one or more hardware engines are generally configured to generate one or more output vectors in response to zero or more input vectors using the operators. At least one of the one or more hardware engines may support input vector dimensions ranging from zero to at least four dimensions. At least one of the one or more hardware engines is implemented solely in hardware.

Flexible hardware engines for handling operating on multidimensional vectors in a video processor

An apparatus includes a memory and a processor. The memory may be configured to store a directed acyclic graph. The processor may be configured to (i) receive a command to run the directed acyclic graph, (ii) parse the directed acyclic graph into a data flow including one or more operators, (iii) schedule the operators in one or more data paths, and (iv) generate one or more output vectors by processing one or more input vectors in the data paths. The processor generally comprises a plurality of hardware engines. The data paths may be implemented with the hardware engines. The hardware engines may operate in parallel to each other.

Computer vision processing in hardware data paths

An apparatus comprises a detector and a processor. The processor may be configured to perform a two-stage object detection process utilizing the detector circuit. The detector circuit may be configured to implement a simple detection stage and a complex detection stage. In the simple detection stage, a first type of object detection is applied to each image in a search space and a number of candidates are identified by applying a first non-maximum suppression technique. In the complex detection stage, a second type of object detection is applied to the candidates identified by the simple detection stage.

Efficient two-stage object detection scheme for embedded device

A method of feature matching is provided. The method includes the steps of (A) determining a set of correspondences between a set of reference feature points for a first image and a set of target feature points for a second image using a processor to compute a descriptor matching function for each pair of reference and target feature points and (B) reducing the set of correspondences using the processor to perform an approximate cross-check to obtain a set of valid correspondences based upon the descriptor matching function computed for each pair of feature points.

Approximate cross-check for real-time feature matching

An apparatus includes a scheduler circuit and a plurality of hardware engines. The scheduler circuit may be configured to (i) store a directed acyclic graph, (ii) parse the directed acyclic graph into a plurality of operators and (iii) schedule the operators in one or more data paths based on a readiness of the operators to be processed. The hardware engines may be (i) configured as a plurality of the data paths and (ii) configured to generate one or more output vectors by processing zero or more input vectors using the operators.

Memory hierarchy to transfer vector data for operators of a directed acyclic graph

An apparatus comprising an interface to a sensor and a processor. The sensor may be configured to generate a video signal having a first field of view based on a targeted view from a vehicle. The processor may be configured to (A) receive the video signal from the interface, (B) generate a second field of view from the video signal based on (i) the first field of view, (ii) a shape of a display and (iii) an input and (C) present the second field of view from the video signal to the display. The second field of view is a cropped version of the first field of view. The second field of view is generated by adjusting video data in the first field of view. Portions of the first field of view not in the second field of view are hidden portions of the video signal.

Apparatus to adjust a field of view displayed on an electronic mirror using an automobile state or a driver action

An apparatus includes an interface and a processor. The interface may be configured to receive an array of scores. The processor may be configured to (i) parse the array of scores into a data flow including one or more operators, (ii) schedule the operators in one or more data paths, (iii) divide the array of scores into blocks of two or more scores by processing the array of scores using the one or more data paths, (iv) compute per-block information using the one or more data paths, and (v) compute local maxima position information for the array of scores by performing block based non-maximum suppression based on the per-block information using the one or more data paths. The data paths are generally implemented with a plurality of hardware circuits.

Block based non-maximum suppression

An apparatus includes a memory and a circuit. The memory may be configured to store an input image having a plurality of color channels. The circuit may be configured to (i) calculate a plurality of average values of each of the color channels in a plurality of windows around each of a plurality of pixel locations in the input image, (ii) calculate a plurality of feature values based on the average values using a feature extraction process and (iii) generate a plurality of likelihood values of a specific color at each of the pixel locations using the feature values in a color classification process.

An apparatus comprises a memory, one or more hardware engines, and a processor. The memory may be configured to store a feature map pyramid comprising an original feature map and a plurality of scaled feature maps. The one or more hardware engines may be configured to perform pooling operations utilizing power of two down sampling. The processor may be configured to generate the feature map pyramid from the original feature map using said one or more hardware engines.

Fast and energy-efficient region of interest pooling for object detection with convolutional neural network

A method of object or feature detection. The method includes the steps of (A) receiving an array of scores and (B) applying a block based non-maximum suppression technique to the array of scores.

An apparatus includes a scheduler circuit and a plurality of hardware engines. The scheduler circuit may be configured to (i) store a directed acyclic graph, (ii) parse the directed acyclic graph into one or more operators and (iii) schedule the one or more operators in one or more data paths. The hardware engines may be (i) configured as a plurality of the data paths and (ii) configured to generate one or more output vectors by processing zero or more input vectors using the operators. One or more of the hardware engines supports a range of multiple dimensions of the input vectors from zero dimensions to at least four dimensions.

Growing community of inventors

Saratoga, CA, United States of America

Leslie D Kohn