Qualcomm Incorporated

Jisoo Lee

Micha Galor Gluskin

Hau Hwang

Vishal I Gupta

Tushar Sinha Pankaj

Ruben Manuel Velarde

Lee-Kang Liu

Kyuseo Han

Jiafu Luo

Leung Chun Chan

Prajit Kulkarni

Techniques are provided for using one or more machine learning systems to process input data including image data. The input data including the image data can be obtained, and at least one machine learning system can be applied to at least a portion of the image data to determine at least one color component value for one or more pixels of at least the portion of the image data. Based on application of the at least one machine learning system to at least the portion of the image data, output image data for a frame of output image data can be generated. The output image data includes at least one color component value for one or more pixels of the frame of output image data. Application of the at least one machine learning system causes the output image data to have a reduced dimensionality relative to the input data.

Image signal processor for processing images

A first image can be processed through a neural network. The first image can cause each first hidden node to assume a first state. A second image can be accessed along with a region-of-interest (ROI) for the second image. At least some of the first hidden nodes can be frozen based on the ROI. At least some of the first hidden nodes can be kept fluid based on the ROI. The second image can be processed through the neural network. The fluid first hidden nodes can assume second states. The frozen first hidden nodes can remain in the first state.

Neural network with frozen nodes

Techniques and systems are provided for providing a rendering engine model for raw media data. In some examples, a system obtains media data captured by a data capturing device and embeds, in a media item containing the media data, a rendering engine model including a description of a neural network configured to process the media data and generate a particular media data output, the description defining a neural network architecture for the neural network. The system then outputs the media item with the rendering engine model embedded in the media item, the rendering engine model indicating how to execute the neural network to process the media data in the media item and generate the particular media data output based on the description of the neural network.

Embedded rendering engine for media data

A method synchronizes autofocus in a system having a master camera and a slave camera. The method comprises: focusing the slave camera based on a map and a result of an autofocus operation by the master camera, while capturing each of a plurality of images. The map relates a plurality of master camera lens positions of the master camera to corresponding slave camera lens positions of the slave camera. An autofocus operation is periodically performed in the slave camera to determine an additional slave camera lens position for an additional image. The map is adaptively updated, based at least partially on the additional slave camera lens position.

Multi-camera autofocus synchronization

An original depth-map of a depth-natural image can be generated. The depth-natural image can include image pixels. The original depth-map can include original depths mapping to one or more of the image pixels. An edge in the depth-natural image can be detected. A supplementary depth can be interpolated based on the original depths and the detected edge. The depth-natural image can be blurred based on the interpolated supplementary depth to generate a depth-artificial image.

Generation of a depth-artificial image by determining an interpolated supplementary depth through interpolation based on the original depths and a detected edge

Techniques and systems are provided for processing image data using one or more neural networks. For example, a patch of raw image data can be obtained. The patch can include a subset of pixels of a frame of raw image data, and the frame can be captured using one or more image sensors. The patch of raw image data includes a single color component for each pixel of the subset of pixels. At least one neural network can be applied to the patch of raw image data to determine a plurality of color component values for one or more pixels of the subset of pixels. A patch of output image data can then be generated based on application of the at least one neural network to the patch of raw image data. The patch of output image data includes a subset of pixels of a frame of output image data, and also includes the plurality of color component values for one or more pixels of the subset of pixels of the frame of output image data. Application of the at least one neural network causes the patch of output image data to include fewer pixels than the patch of raw image data. Multiple patches from the frame can be processed by the at least one neural network in order to generate a final output image. In some cases, the patches from the frame can be overlapping so that the final output image contains a complete picture.

A method for image capture control may include computing an electronic image stabilization (EIS) compensation based on image data from an image capture device, the image data having a first region of interest (ROI). A jitter or panning shift is computed based on the EIS compensation. The first ROI is adjusted for use in at least one of the group consisting of an automatic white balance process and an automatic exposure process of the image capture device based on the jitter or panning shift.

Image capture device with stabilized exposure or white balance

A scene can be captured by integrating a first sensor pixel for a first amount of time to produce an original first photometric and integrating a second sensor pixel for the first amount of time to produce an original second photometric. The first sensor pixel can be configured to saturate with photocharge slower than the second sensor pixel. The scene can be recaptured by integrating the second sensor pixel for a second amount of time less than the first amount of time.

Dynamic range estimation with fast and slow sensor pixels

Aspects of the present disclosure relate to systems and methods for hybrid depth map processing. An example device may include a first sensor having a plurality of focus pixels, the focus pixels configured to capture images from at least a first perspective and a second perspective, where a difference between the first and second perspectives is associated with a first direction. The device may further include a second image sensor separated from the first image sensor in a second direction orthogonal to the first direction. The device may be configured to receive a first image from the first image sensor, where the first image includes first image data captured from the first perspective, and second image data captured from the second perspective. The device may be further configured to receive a second image from the second image sensor, and to generate a hybrid depth map based at least in part on the first image and the second image.

Hybrid depth processing

Methods, devices, and systems for automatic white balance are disclosed. In some aspects, a device includes a memory and a processor coupled to a camera and the memory. The processor may be configured to determine a first number of statistics associated with a first frame, cause the camera to perform a first autofocus operation, determine a second number of statistics associated with a second frame during the first autofocus operation, the second number of statistics being less than the first number of statistics, determine a white balance gain based on the first number of statistics and the second number of statistics, and apply the white balance gain to a third frame.

Auto focus based auto white balance

Methods, systems, and apparatuses are provided to perform phase-detection autofocus control. By way of example, the methods can receive luminance values measured by a plurality of sensing elements in a sensor array, and the sensing elements can include imaging pixels and phase-detection pixels. The methods can compare luminance values measured by at least one of the phase-detection pixels to luminance values associated with a subset of the imaging pixels including two or more imaging pixels. The comparison can be performed at extended horizontal density or full horizontal density along a first sensor-array row that includes the at least one phase-detection pixel and the two or more imaging pixels. The methods can also perform a phase-detection autofocus operation based on an outcome of the comparison.

Extended or full-density phase-detection autofocus control

Methods and apparatuses for calibration of phase detection auto focus (PDAF) camera systems are disclosed. In one aspect, the method involves capturing a first image of a scene at an initial lens position, the first image including a first left image and a first right image captured using left and right photodiodes. The method may also involve calculating an initial phase difference between the first left image and first right image and estimating an in-focus lens position based on the initial phase difference. The method may further involve moving the lens to a final lens position at which a final phase difference between a second left image and a second right image of a second image captured at the final lens position is substantially zero and calibrating the estimation of the in-focus lens position based on the initial lens position, the final lens position, and the initial phase difference.

Calibration for phase detection auto focus (PDAF) camera systems

A method provides phase disparity data. First phase disparity data for a first image frame of a first scene is computed using a phase disparity engine of an image capture device. The method determines whether the image capture device is receiving light from a second scene that is different from the first scene and corresponds to a second image frame. The phase disparity engine discontinues computing of phase disparity data, in response to a determination that the second scene is not different from the first scene. The first phase disparity data are output in connection with the second image frame, in response to determining that the second scene is not different from the first scene.

Growing community of inventors

San Diego, CA, United States of America

Jisoo Lee