Apple Inc.

Xerox Corporation

Aptina Imaging Corporation

Semiconductor Components Industries, LLC

Yingjun Bai

Xuemei Zhang

Xing Li

Ramesh Nagarajan

Zhigang Fan

Touraj Tajbakhsh

Gregory Guyomarc'h

Francis Kapo Tse

Paul M Hubel

Haitao Guo

Yonghui Zhao

Vignesh Jagadeesh

Mark N Gamadia

Qun Sun

Michael Chin

Juanqing Chen

Guillaume Tartavel

D Amnon Silverstein

Hao Pan

David Jon Metcalfe

Gennadiy A Agranov

Moshe Laifenfeld

Clara Cuciurean-Zapan

Xiangli Li

Todd Stephen Sachs

Farhan A Baqai

Marius Tico

Michael Robert Campanelli

Muge Wang

Timothy Wayne Jacobs

Zhenhuan Wen

Brian W Keelan

David Daming Kuo

Xiaohua Yang

Peter D McCandlish

George E Williams

Brandon J Corey

Fabio Riccardi

John Christopher Cassidy

Scott A Bennett

Stephen F Linder

Changmeng Liu

Yongfang Liang

Honghong Peng

Munehiro Nakazato

Henryk K Blasinski

Elizabeth Torres Martin

Miguel Flores

Andrew Hanvey

Kosta Dabov

Zhongmin Wang

David Kuo

Various embodiments disclosed herein include techniques for determining autofocus for a camera on a mobile device. In various instances, a depth imaging system (e.g., a time-of-flight autofocus system (ToF-AF system)) is used to determine distance of a subject in order to determine autofocus for a camera. In some instances, however, the ToF-AF system may be unable to detect a subject that is very close to the camera when the objects are below a minimum detectable distance of the ToF-AF system. In such instances, an existing IR detector outside of the ToF-AF system may be implemented to measure reflected signals from the ToF-AF system. A power ratio may be determined from the reflected signals and used to determine information about the distance of the subject from the camera.

Near distance detection for autofocus

Various methods for utilizing a saliency heatmaps are described. The methods include obtaining image data corresponding to an image of a scene, obtaining a saliency heatmap for the image of the scene based on a saliency network, wherein the saliency heatmap indicates a likelihood of saliency for a corresponding portion of the scene, and manipulating the image data based on the saliency heatmap. In embodiments, the saliency heatmap may be produced using a trained machine learning model. The saliency heatmap may be used for various image processing tasks, such as determining which portion(s) of a scene to base an image capture device's autofocus, auto exposure, and/or white balance operations upon. According to some embodiments, one or more bounding boxes may be generated based on the saliency heatmap, e.g., using an optimization operation, which bounding box(es) may be used to assist or enhance the performance of various image processing tasks.

Saliency of an object for image processing operations

An electronic image capture device captures a first image of a scene at a first time. A first local tone mapping operator for a first portion of the first image is determined. The electronic image capture device further captures a second image of the scene at a second time. A motion of the electronic device between the first time and the second time is determined. A second local tone mapping operator for a second portion of the second image is determined. The second portion is determined to correspond to the first portion based, at least in part, on the determined motion of the electronic device. The second local tone mapping operator is determined based, at least in part, on the first local tone mapping operator. At least the second local tone mapping operator is applied to the second portion of the second image.

Camera motion aware local tone mapping

Determining a focus setting includes determining a plurality of regions of interest in a view of a scene, and, for each of the plurality of regions of interest, obtaining a set of image data for each of multiple focal positions, and then applying focus filters to the set of image data for each of the plurality of focal positions for each of the regions of interest to obtain a set of focus scores, i.e., a focus score for each focus filter applied to the set of image data for each of the focal positions. Further, determining a confidence value associated with each of the sets of focus scores, selecting a subset of the sets of focus scores based on the confidence values associated with each of the sets of focus scores, and determining a focus setting for the scene based on the selected subset of the focus scores.

Method and system for robust contrast based auto focus in low light

Techniques for auto white balancing of captured images based on detection of flicker in ambient light is described. When flicker is detected in ambient light during an image capture event, and the flicker is unchanging during the image capture event, a white point of image data may be estimated according to a first technique. When flicker is detected in ambient light during an image capture event, and the flicker is changing during the image capture event, a white point of image data may be estimated according to a second technique. When flicker is not detected, a white point of image data may be estimated according to a third technique. Image data may be color corrected based on the estimated white point.

Auto white balance control algorithm based upon flicker frequency detection

This disclosure relates to a wide gamut encoder capable of receiving a wide gamut color image in accordance with a wide gamut standard. The encoder can encode one or more wide gamut color image pixel values into portions of narrow gamut encoding elements for transmission to a video encoder. The encoder can implement an advanced extended YCC format that is backward compatible with a P3 color gamut.

Extended YCC format for backward-compatible P3 camera video

This disclosure pertains to systems, methods, and computer readable media for performing lens shading correction (LSC) operations that modulate gains based on scene lux level and lens focus distance. These gains compensate for both color lens shading (i.e., the deviation between R, G, and B channels) and vignetting (i.e., the drop off in pixel intensity around the edges of an image). As scene illuminance increases, the sensor captures more signal from the actual scene, and the lens shading effects begin to appear. To deal with the situation, the lens shading gains are configured to adaptively 'scale down' when scene lux approaches zero and 'scale up' when scene lux changes from near zero to become larger. The lens shading gain may also be modulated based on the focus distance. For optical systems without zoom, the inventors have discovered that the amount of lens shading fall off changes as focus distance changes.

Lens shading modulation

Displaying wide-gamut images as intended on color-managed wide-gamut display systems while rendering a visually consistent image when rendered on targeted narrow-gamut display systems (regardless of whether the narrow-gamut displays are color-managed). Images represented in accordance with this disclosure are referred to as a dual-target images (DTI): one target being the image's original wide-gamut color space, the other target being a specified narrow-gamut color space. The novel representational scheme retains narrow-gamut rendering for those colors in a wide-gamut image that are within the targeted narrow-gamut color space, transitioning to wide-gamut rendering for those colors in the wide-gamut image that are outside the targeted narrow-gamut color space. This approach minimizes pixel clipping when rendering a wide-gamut image for a narrow-gamut display, while allowing the original wide-gamut pixel values to be recovered when rendering for a wide-gamut display.

Dual-target image color rendering

An image system with a dual layer photodiode structure is provided for processing color images. In particular, the image system can include an image sensor that can include photodiodes with a dual layer photodiode structure. In some embodiments, the dual layer photodiode can include a first layer of photodiodes (e.g., a bottom layer), an insulation layer disposed on the first layer of photodiodes, and a second layer of photodiodes (e.g., a top layer) disposed on the insulation layer. The first layer of photodiodes can include one or more suitable pixels (e.g., green, blue, clear, luminance, and/or infrared pixels). Likewise, the second layer of photodiodes can include one or more suitable pixels (e.g., green, red, clear, luminance, and/or infrared pixels). An image sensor incorporating dual layer photodiodes can gain light sensitivity with additional clear pixels and maintain luminance information with green pixels.

Image sensor with dual layer photodiode structure

A gamut size metric is used in all phases of color image processing (e.g., capture, transmission, and display). In general, the gamut size metric is a single-valued metric that changes as image content changes. More particularly, a gamut boundary histogram is determined and used to estimate a gamut size metric. A gamut size metric identifies a minimum size gamut needed to encompass each pixel in an image, where the gamut size is limited at one end by a first device independent gamut (S), and at another end by a second device independent color space (S), where Sis wholly enclosed within S. The gamut size metric may be based on strict pixel color value differences. In other embodiments the gamut size metric may take into effect perceptual color differences and significance.

Color gamut size metric estimation

Methods, devices and computer readable instructions to generate region-of-interest (ROI) tone curves are disclosed. One method includes obtaining a statistic for an entire image such as, for example, a luminance statistic. The same statistic may then be found for a specified ROI of the image. A weighted combination of the statistic of the entire image and the statistic of the ROI yields a combined statistic which may then be converted to a ROI-biased tone curve. The weight used to combine the two statistics may be selected to emphasize or de-emphasize the role of the ROI's statistic in the final tone curve.

Region-of-interest biased tone mapping

A system and method convert a pixel of binary image data to a pixel of contone image data by determining if a predetermined pixel of binary image data is part of a solid edge or part of a fuzzy edge. A binary to contone conversion circuit converts the predetermined pixel of binary image data to a pixel of a first contone image data value, and a filter circuit converts the predetermined pixel of binary image data to a pixel of a second contone image data value. The filter circuit uses an adaptive filtering operation wherein the adaptive filtering operation utilizes one of a plurality of sets of weighting coefficients to change a characteristic of the filtering operation. The set of weighting coefficients used in the filtering operation are selected in response to a fuzzy edge detection. A selection between the first contone image data value and the second contone image data value is made based upon the determination as whether the predetermined pixel of binary image data is part of a solid edge.

Method for binary to contone conversion with non-solid edge detection

This disclosure pertains to systems, methods, and computer readable media for extending the dynamic range of images using an operation referred to herein as 'Adaptive Auto Exposure' (AAE). According to the embodiments disclosed herein, the AAE-enabled higher dynamic range capture operations are accomplished without blending multiple or bracketed exposure captures (as is the case with traditional high dynamic range (HDR) photography). AAE also enables high signal-to-noise ratio (SNR) rendering when scene content allows for it and/or certain highlight clipping is tolerable. Decisions with regard to preferred AE strategies may be based, at least in part, on one or more of the following: sensor characteristics; scene content; and pre-defined preferences under different scenarios.

Adaptive auto exposure and dynamic range compensation

Growing community of inventors

San Jose, CA, United States of America

Yingjun Bai