Broadcom Corporation

Apple Inc.

Sony Corporation

Sony Electronics Inc

Jaewon Shin

Brian Francis Schoner

Darren Duane Neuman

Advait Madhav Mogre

Suk Hwan Lim

Christopher L Mills

Richard Hayden Wyman

Fernando Alberto Mujica

Liran Fishel

Sung Hee Park

Erik K Norden

Seungjin Lee

Ali Jezu Tabatabai

D Amnon Silverstein

Mohammad Gharavi-Alkhansari

David Chaohua Wu

Flaviu Dorin Turean

Hyeong-Seok Victor Ha

Olivier Francois Joseph Harel

Embodiments relate to a neural processor circuit with scalable architecture for instantiating one or more neural networks. The neural processor circuit includes a data buffer coupled to a memory external to the neural processor circuit, and a plurality of neural engine circuits. To execute tasks that instantiate the neural networks, each neural engine circuit generates output data using input data and kernel coefficients. A neural processor circuit may include multiple neural engine circuits that are selectively activated or deactivated according to configuration data of the tasks. Furthermore, an electronic device may include multiple neural processor circuits that are selectively activated or deactivated to execute the tasks.

Scalable neural network processing engine

Embodiments relate to a histogram-of-oriented gradients (HOG) module. The HOG module is implemented in hardware rather than software. The HOG module applies an algorithm to an image to identify gradient orientation in localized portions of the image. The HOG module creates a histogram-of orientation gradients based on the identified gradient orientations.

Configurable histogram-of-oriented gradients (HOG) processor

A cross fader circuit that receives first raw image data and second raw image data and outputs blended raw image data. The cross fader circuit includes a first scaling circuit, a second scaling circuit, and an alpha blender. The first scaling circuit downscales first raw image data captured by a first sensor with a first field of view to match a size of a blending window. The second scaling circuit upscales second raw image data to match the size of a canvas window that encloses the blending window. The second raw image data may be a cropped version of raw image data captured by a second sensor of a second field of view wider than the first field of view. An alpha blender circuit generates a blended raw image data matching the size of the canvas window from the downscaled first raw image data and upscaled second raw image data.

Systems and methods for crossfading image data

Embodiments relate to a patch processor that warps patches of input image data. The patch processor includes a patch direct memory access (DMA) circuit that obtains the patches via direct memory access. The patch processor includes a patch warper circuit that generates warped patches by processing the patches by performing interpolation in a raster scan fashion using a set of coordinates, for example. The patch warper circuit may also process pixels of the patches using an adder or subtractor circuit. In addition, the patch warper circuit may interleave warped patches of different image channels such as RGB or YCbCr colors. The patch warper circuit can also double-buffer the patches and warped patches.

Patch warper circuit for image processing

Embodiments of the present disclosure relate to a sensor interface circuit that performs scaling of image data in a Bayer pattern without spreading defective pixels across multiple pixels. The sensor interface circuit may include a register circuit storing operating parameters of the sensor interface circuit. The sensor interface circuit includes a scaling circuit with a first defect pixel detection circuit to detect a first defective pixel in an input image by analyzing pixels in a line of an input image data along a first direction. A first scaling circuit is coupled to the first defect pixel detection circuit and generates a scaled line of pixels representing the line of the input image scaled along the first direction according to the operating parameters.

Scaling of image data in sensor interface based on detection of defective pixels

Systems and methods for upscaling graphics resolution are provided. The systems and methods may selectively upscale graphics frames having a first resolution to a second resolution by various scaling methods. Depending on a variety of factors, at least a first mode of upscaling and a second mode of upscaling may provide for improved upscaling on a variety of graphics data.

System and method for graphics upscaling

A video processing system may be operable to utilize multi-band sharpening to process luma signals for image signals. The luma signal may be decomposed into a plurality of frequency band components, wherein each component may be processed separately using different sharpening gains and/or offsets. The multi-band processed components may be combined to generate sharpened output luma signals. The multi-band sharpening may be performed utilizing peaking processing, and the input luma signal and/or LTI sharpened luma signals may be combined with the multi-band peaking sharpened signals to generate the sharpened output luma signals. Corresponding chroma signals may also be adjusted to generate sharpened output chroma signals. Luma and/or chroma sharpening operations may be further adjusted based on coring, clipping avoidance, luma statistics, color region detections, and/or curve control parameters. Sharpened output image signals may be generated based on the sharpened output luma signals and the sharpened output chroma signals.

Method and system for sharpening the luma and the chroma signals

Aspects of an edge smoothing block filter and combinations of the filter with a motion compensated temporal filter are described. In one embodiment, edge smoothing block filtering includes selecting a current pixel to be filtered and selecting a candidate pixel within a search area about the current pixel. The edge smoothing block filtering generally seeks to identify candidate pixels having surrounding pixels similar to pixels surrounding the current pixel. The edge smoothing block filtering further computes a cost difference between pixels within a candidate pixel neighborhood and pixels within a current pixel neighborhood, and filters the current pixel based on the cost difference. Aspects of the filters and filter element combinations may preserve edges and textures adaptively based on image content. For example, diagonal or curved image edges may be filtered along edges while texture is preserved along the edges.

Edge smoothing block filtering and blending

A method for providing hardware mandatory access control (MAC) may include coupling an input filter to a hardware core. The input filter may receive one or more labeled objects and may qualify the labeled objects based on a first label associated with each of the labeled objects. The input filter may strip the first label from each of the labeled objects to provide one or more input objects. The hardware core may receive the one or more input objects and may provide one or more output objects. An output filter may be coupled to the hardware core. The output filter may receive the one or more output objects and may associate to each of the one or more output objects a second label. Operations of the input filter and the output filter may be coordinated by a label manager.

Method for mandatory access control of processing in hardware cores

A method for seamless transition between interlaced and progressive video profiles may include receiving at a video feeder a stream of video signals for displaying on a display device. The stream of video signals may include segments of one of interlaced fields or progressive video frames. During an interlaced stream, the interlaced fields may be directed to a deinterlacer, and an output of the deinterlacer may be directed to the display device. The output of the deinterlacer may be determined based on previously received interlaced fields from the video feeder. During a transition to a progressive stream, the progressive video frames may be directed to a frame store, and the output of the deinterlacer may be directed to the display device. During a progressive stream, the progressive video frames may be directed to the frame store, and previously stored frames may be directed to the display device.

Seamless transition between interlaced and progressive video profiles in an ABR system

A method and system are provided in which a processor, such as a video processor, may determine a flatness value for a current video picture and may adjust a mosquito noise filtering of a subsequent video picture based on the determined flatness value. The flatness value may be determined within a predetermined region of the current video picture, which may be dynamically modified by the processor. The flatness value may be associated with the presence of analog noise in the current video picture, and may be determined based on horizontal and vertical variance values determined for one or more portions of the current video picture. The processor may adjust a mosquito noise strength value of the current video picture based on the determined flatness value and may adjust the mosquito noise filtering of the subsequent video picture based on the adjusted mosquito noise strength value of the current video picture.

Method and system for detecting analog noise in the presence of mosquito noise

In a video processing device, scale of a video image is detected for vertical and horizontal directions based on pixel information, for example, per pixel vertical and horizontal gradients. Gradients are utilized or discarded based on picture format, standard deviation of luma levels and pixel location relative to black border edges, graphics and/or overlaid content. Mosquito noise filters are adapted based on scale and/or noise strength. Median and/or linear filter results are selected based on a weakest, a strongest and/or a blended result. Horizontal and vertical operations are performed separately for edge detection, edge strength determination, filtering and filter correction control. Horizontal and vertical block grid spacing and grid shift are determined. Block noise strength is determined. Block noise filters are configured based on scaling and/or noise strength. Filter corrections are limited based on block noise strength. Noise reduction results may be blended to generate a pixel correction value.

Method and system for digital noise reduction of scaled compressed video pictures

Methods and systems for processing chroma signals are disclosed. Aspects of one method may include compensating a modulated chroma signal for distortions due to audio trap filtering. The compensated chroma signal may be demodulated to generate U and V chroma signals. The U and V chroma signals may be low pass filtered, and then YC aligned. YC alignment may comprise correcting portions of the U and V chroma signals based on the luma signal. The correction may be based on, for example, correlation between the U and V chroma signals and the Y luma signal, and/or energy level of the Y luma signal. The corrected U and V chroma signals may be chroma sharpened, where a gain parameter may be changed to change a transition slope for the U and V chroma signals. The chroma sharpened signals may also be clamped to reduce overshoot and undershoot.

Growing community of inventors

Mountain View, CA, United States of America

Jaewon Shin