Ambarella, Inc.

Lsi Corporation

International Business Machines Corporation

Lsi Logic Corporation

C-cube Microsystems, Inc.

Ambarella International LP

C-cube Semiconductor Ii Inc.

Harmonic, Inc.

Ambrella, Inc.

Elliot N Linzer

Leslie D Kohn

Lowell Leroy Winger

Aaron G Wells

Ho-Ming Leung

Guy Cote

Aaron Wells

Yunwei Jia

Simon Booth

Ephraim Feig

Chi Hong John Ju

Cesar Augusto Gonzales

Didier LeGall

Wilson Kwok

Hsiang-Chun Huang

Yu Wang

Eric Viscito

Guy Rapaport

Prasoon Tiwari

Heidi Aarlien Peterson

Ian R Finlay

Agnes Yee Ngai

Peter Hans Westerink

Eric C Pearson

Roderick Michael West

Thomas A Horvath

Charlene Ann Gebler

John Ju

Xiaoming Li

Soo-Chul Han

Cecelia L Dopp

Dzung Tien Hoang

An apparatus includes a convolutional neural network comprising (i) a first convolution having a plurality of input frames and (ii) a last convolution having a plurality of output frames. Each output frame generally represents a different instance in time.

Convolutional neural network with multiple output frames

An apparatus includes a memory and a processor circuit. The memory may be configured to store one or more frames of image pixel data. Each of the frames generally comprises red (R) samples, green (G) samples, blue (B) samples, and infrared (IR) samples. The processor circuit may be configured to generate an infrared image for each frame. The infrared image generally has a number of infrared (IR) pixels greater than the number of the infrared (IR) samples of each frame. The processor circuit generally performs interpolation utilizing the infrared (IR) samples and one or more of the red (R) samples, the green (G) samples, and the blue (B) samples of each frame in generating the infrared image for each frame.

High resolution infrared image generation using image data from an RGB-IR sensor and visible light interpolation

An apparatus includes a raw image pipeline comprising a first circuit and a second circuit. The first circuit of the raw image pipeline may be configured to distinguish between smooth picture noisy areas and non-smooth clean areas of an image by performing tone based non-smooth detection on data of the image to obtain an adjusted non-smoothness value for at least one area comprising a plurality of pixels of the image. The second circuit of the raw image pipeline may be configured to adjust one or more of noise reduction filtering or sharpening filtering performed on the at least one area of the image based on the adjusted non-smoothness value.

Tone based non-smooth detection

An apparatus includes a first circuit and a second circuit. The first circuit may be configured to (i) generate a sharpened region by sharpening an original region of an image and (ii) generate edge information by detecting an edge in the original region of the image. The second circuit may be configured to (i) evaluate a current sample of the sharpened region proximate to the edge for an artifact based on the edge information and (ii) generate a final sample by adjusting the current sample in the sharpened region where the artifact is detected. The artifact may be an overshoot or an undershoot.

Image sharpening with edge direction based undershoot and overshoot reduction

A method of noise filtering an image is presented. The method includes the steps of (A) computing a non-smooth amount for at least one area of the image, (B) computing a tone value for the at least one area of the image, (C) computing an adjusted non-smooth amount for the at least one area by adjusting the non-smooth amount for the at least one area of the image based on the tone value for the at least one area of the image, and (D) performing one or more image processing steps on the at least one area of the image based on the adjusted non-smooth amount.

An apparatus includes an interface and a processor. The interface may be configured to receive an array of scores. The processor may be configured to (i) parse the array of scores into a data flow including one or more operators, (ii) schedule the operators in one or more data paths, (iii) divide the array of scores into blocks of two or more scores by processing the array of scores using the one or more data paths, (iv) compute per-block information using the one or more data paths, and (v) compute local maxima position information for the array of scores by performing block based non-maximum suppression based on the per-block information using the one or more data paths. The data paths are generally implemented with a plurality of hardware circuits.

Block based non-maximum suppression

An apparatus includes a first circuit and a second circuit. The first circuit may be configured to (a) receive (i) a plurality of first samples selected from a first picture of a sequence of pictures, (ii) a plurality of smooth first samples corresponding to the plurality of first samples, (iii) a plurality of second samples selected from a second picture of the sequence of pictures, and (iv) a plurality of smooth second samples corresponding to the plurality of second samples, and (b) generate a plurality of adjusted first samples by combining the smooth first samples, the first samples, and the smooth second samples. The second circuit may be configured to generate a plurality of values based on the plurality of adjusted first samples and at least one of the plurality of first samples and the plurality of second samples. The first samples and the second samples generally have different levels of a lighting condition. The adjusted first samples and the second samples generally have similar levels of the lighting condition.

Lighting correction for motion detection and temporal filtering

An apparatus includes a processor circuit and a correction circuit. The processor circuit may be configured to receive a sequence of pictures. The correction circuit may be configured to (i) generate a motion score based upon target samples from a target picture and reference samples from one or more reference pictures of the sequence of pictures, (ii) generate a first weight value based upon the motion score and a first weight limit value, and (iii) generate a plurality of blended picture samples by combining the target samples and the reference samples based upon the first weight value.

Temporal filter with criteria setting maximum amount of temporal blend

A method for temporal filtering based on motion detection between non-adjacent pictures. The method may compute a motion score by motion detection between a target area in a target picture and a first area in a non-adjacent one of a plurality of reference pictures; and temporal filter the target area with a second area in an adjacent one of the reference pictures based on the motion score to generate a filtered area in a filtered picture. At least one of (i) the motion score and (ii) the generation of the filtered area may be controlled by one or more gain settings in a circuit.

Temporal filtering based on motion detection between temporally non-adjacent pictures

A method of object or feature detection. The method includes the steps of (A) receiving an array of scores and (B) applying a block based non-maximum suppression technique to the array of scores.

An apparatus includes a processor circuit and a correction circuit. The processor circuit may be configured to receive a sequence of pictures. The correction circuit may be configured to (i) generate multiple smooth first samples by spatially smoothing multiple first samples selected from a first picture of said sequence of pictures, (ii) generate multiple smooth second samples by spatially smoothing multiple second samples selected from a second picture of said sequence of pictures, (iii) generate multiple adjusted first samples by combining said smooth first samples, said first samples and said smooth second samples, and (iv) generate multiple values based on said second samples and said adjusted first samples. The first samples and the second samples may have different levels of a lighting condition. The adjusted first samples and the second samples may have similar levels of the lighting condition.

A method for temporal filtering based on motion detection on areas of different sizes includes computing a plurality of motion scores of a plurality of areas in a target picture in a sequence of pictures by motion detection of the areas between the target picture and a reference picture in the sequence of pictures, where the areas may be at least two overlapping areas of different sizes, computing a combined motion score by combining the plurality of motion scores, and generating a filtered picture by temporal filtering the target picture with the reference picture based on the combined motion score. At least one of (i) the motion scores and (ii) the filtered picture may be based on one or more gain settings in a circuit.

Temporal filtering based on motion detection on areas of different sizes

A method for motion detection based on observing several pictures. The method may include computing a first motion score of an area in a target picture of a sequence of pictures by a comparison of the area between the target picture and a first reference picture; computing a second motion score of the area by another comparison of the area between a second reference picture and a third reference picture, where the target picture is a different picture than the second reference picture and the third reference picture; and temporal filtering the target picture with the first reference picture based on the first motion score and the second motion score. At least one of the computing of the first motion score, the computing of the second motion score, and the temporal filtering is controlled by one or more gain settings applied to the sequence of pictures in a circuit.

Motion detection based on observing several pictures

An apparatus includes an input circuit and a processing circuit. The input circuit may be configured to receive a sequence of pictures. The processing circuit may be configured to (i) generate a change factor based on gain information for a reference picture selected from the sequence of pictures, gain information for a target picture selected from the sequence of pictures, an average pixel value for the reference picture, and an average pixel value for the target picture, (ii) remap data of a selected one of the reference picture and the target picture by calculating pixel values for the selected picture using original pixel values of the selected picture and the change factor, and (iii) perform motion detection between the reference picture and the target picture utilizing (a) the remapped image data of the selected picture and (b) original data of the picture that was not remapped.

Picture brightness adjusted motion detection

A method for motion detection based on observing several pictures is disclosed. Step (A) may compute a first motion score of an area in a target picture by a comparison of the area between the target picture and a first reference picture. Step (B) may compute a second motion score of the area by another comparison of the area between the target picture or a second reference picture and a third reference picture. Step (C) may temporal filter the target picture with the first reference picture based on the first motion score and the second motion score. At least one of the computing of the first motion score, the computing of the second motion score, and the temporal filtering may be controlled by one or more gain settings in a circuit. At least two of the first, the second, and the third reference pictures may be different pictures.

Growing community of inventors

Suffern, NY, United States of America

Elliot N Linzer