Apple Inc.

Hewlett-Packard Development Company, L.p.

Agilent Technologies, Inc.

Micron Technology Incorporated

Avago Technologies Ecbu IP (Singapore) Pte. Ltd.

Aptina Imaging Corporation

Xuemei Zhang

Yingjun Bai

Ramakrishna Kakarala

Izhak Baharav

Dietrich Werner Vook

Ilia Ovsiannikov

Dmitri Jerdev

Richard Lawrence Baer

Daniel R Tretter

Paul M Hubel

Yonghui Zhao

Farhan A Baqai

Todd Stephen Sachs

Marius Tico

Xiaohua Yang

C Brian Atkins

John H Stanback

Paul Lewis Corredoura

Fabio Riccardi

Juanqing Chen

Aibo Tian

D Amnon Silverstein

Hui Chao

Hao Pan

Haitao Guo

Graham D Finlayson

Claus Molgaard

Travis N Blalock

Wei Sun

S Jeffrey Rosner

David Daming Kuo

George E Williams

Clayton Atkins

Brandon J Corey

Gregory Guyomarc'h

James Mahon

Jun Hu

Steven David Hordley

Jun Xiao

Munehiro Nakazato

Yongfang Liang

Yuli Gao

Donald M Reid

Douglas Sojourner

Henryk K Blasinski

Robert W Jones

Christophe Seyve

Phil Cheatle

Bond Yu-Pong Ying

Russell M Iimura

Jeanne M Hermsen

Harold F Christensen, Jr

Jin Wook Chang

Kosta Dabov

Bond Yu Pong Ying

Feng Xiao

Yingmei Lavin

David H Brainard

In one implementation, a method is performed for generating metadata estimations based on metadata subdivisions. The method includes: obtaining an input image; obtaining metadata associated with the input image; subdividing the metadata into a plurality of metadata subdivisions; determining a viewport relative to the input image based on at least one of head pose information and eye tracking information; generating one or more metadata estimations by performing an estimation algorithm on at least a portion of the plurality of metadata subdivisions based on the viewport; and generating an output image by performing an image processing algorithm on the input image based on the one or more metadata estimations.

Method and device for generating metadata estimations based on metadata subdivisions

Devices, methods, and non-transitory program storage devices are disclosed herein to perform intelligent determinations of non-linear (i.e., dynamic) image recording rates for the production of improved timelapse videos. The techniques described herein may be especially applicable to timelapse videos captured over long durations of time and/or with varying amounts of device motion/scene content change over the course of the captured video (e.g., when a user is walking, exercising, driving, etc. during the video's capture). By smoothly varying the image recording rate of the timelapse video in accordance with multi-temporal scale estimates of scene content change, the quality of the produced timelapse video may be improved (e.g., fewer long stretches of the video with too little action, as well as fewer stretches of the video where there is so much rapid action in the timelapse video that it is difficult for a viewer to perceive what is happening in the video).

Non-linear timelapse videos based on multi-temporal scale detection of scene content change

The disclosure pertains to techniques for image processing. One such technique comprises a method for image selection, comprising obtaining a sequence of images, detecting one or more objects in one or more images of the sequence of images, determining a location for each detected object in the one or more images, determining a trajectory for each detected object based on a determined location for each respective detected object in two or more images of the sequence of images, determining a trajectory waypoint score for the trajectory of each detected object, determining a set of selected images based on an aggregation of trajectory waypoint scores for each detected object in each respective image, and outputting the set of selected images for presentation.

Motion trajectory tracking for action detection

An electronic image capture device captures a first image of a scene at a first time. A first local tone mapping operator for a first portion of the first image is determined. The electronic image capture device further captures a second image of the scene at a second time. A motion of the electronic device between the first time and the second time is determined. A second local tone mapping operator for a second portion of the second image is determined. The second portion is determined to correspond to the first portion based, at least in part, on the determined motion of the electronic device. The second local tone mapping operator is determined based, at least in part, on the first local tone mapping operator. At least the second local tone mapping operator is applied to the second portion of the second image.

Camera motion aware local tone mapping

Techniques for auto white balancing of captured images based on detection of flicker in ambient light is described. When flicker is detected in ambient light during an image capture event, and the flicker is unchanging during the image capture event, a white point of image data may be estimated according to a first technique. When flicker is detected in ambient light during an image capture event, and the flicker is changing during the image capture event, a white point of image data may be estimated according to a second technique. When flicker is not detected, a white point of image data may be estimated according to a third technique. Image data may be color corrected based on the estimated white point.

Auto white balance control algorithm based upon flicker frequency detection

This disclosure relates to a wide gamut encoder capable of receiving a wide gamut color image in accordance with a wide gamut standard. The encoder can encode one or more wide gamut color image pixel values into portions of narrow gamut encoding elements for transmission to a video encoder. The encoder can implement an advanced extended YCC format that is backward compatible with a P3 color gamut.

Extended YCC format for backward-compatible P3 camera video

An image sensor device has a first region configured to sense only infrared illumination and a second region configured to not sense visible and infrared illumination.

Ambient infrared detection in solid state sensors

Displaying wide-gamut images as intended on color-managed wide-gamut display systems while rendering a visually consistent image when rendered on targeted narrow-gamut display systems (regardless of whether the narrow-gamut displays are color-managed). Images represented in accordance with this disclosure are referred to as a dual-target images (DTI): one target being the image's original wide-gamut color space, the other target being a specified narrow-gamut color space. The novel representational scheme retains narrow-gamut rendering for those colors in a wide-gamut image that are within the targeted narrow-gamut color space, transitioning to wide-gamut rendering for those colors in the wide-gamut image that are outside the targeted narrow-gamut color space. This approach minimizes pixel clipping when rendering a wide-gamut image for a narrow-gamut display, while allowing the original wide-gamut pixel values to be recovered when rendering for a wide-gamut display.

Dual-target image color rendering

A gamut size metric is used in all phases of color image processing (e.g., capture, transmission, and display). In general, the gamut size metric is a single-valued metric that changes as image content changes. More particularly, a gamut boundary histogram is determined and used to estimate a gamut size metric. A gamut size metric identifies a minimum size gamut needed to encompass each pixel in an image, where the gamut size is limited at one end by a first device independent gamut (S), and at another end by a second device independent color space (S), where Sis wholly enclosed within S. The gamut size metric may be based on strict pixel color value differences. In other embodiments the gamut size metric may take into effect perceptual color differences and significance.

Color gamut size metric estimation

Methods, devices and computer readable instructions to generate region-of-interest (ROI) tone curves are disclosed. One method includes obtaining a statistic for an entire image such as, for example, a luminance statistic. The same statistic may then be found for a specified ROI of the image. A weighted combination of the statistic of the entire image and the statistic of the ROI yields a combined statistic which may then be converted to a ROI-biased tone curve. The weight used to combine the two statistics may be selected to emphasize or de-emphasize the role of the ROI's statistic in the final tone curve.

Region-of-interest biased tone mapping

Image enhancement is achieved by separating image signals, e.g. YCbCr image signals, into a series of frequency bands and performing locally-adaptive noise reduction on bands below a given frequency but not on bands above that frequency. The bands are summed to develop the image enhanced signals. The YCbCr, multi-band locally-adaptive approach to denoising is able to operate independently&#x2014;and in an optimized fashion&#x2014;on both luma and chroma channels. Noise reduction is done based on models developed for both luma and chroma channels by measurements taken for multiple frequency bands, in multiple patches on the ColorChecker chart, and at multiple gain levels, in order to develop a simple yet robust set of models that may be tuned off-line a single time for each camera and then applied to images taken by such cameras in real-time without excessive processing requirements and with satisfactory results across illuminant types and lighting conditions.

Multi-band YCbCr locally-adaptive noise modeling and noise reduction based on scene metadata

Methods, devices and computer readable instructions to generate multi-scale tone curves are disclosed. One method includes finding, for a given input image, a global tone curve that exhibits monotonic behavior. The input image may then be partitioned into a first number of sub-regions. For each sub-region, a local tone curve may be determined that has an output level that is constrained to the global tone curve at one or more first luminance levels so that each sub-region's local tone curve's output follows the global tone curve's monotonic behavior. If the resulting local tone curves provide sufficient control of shadow-boost, highlight-suppression, and contrast optimization the first number of local tone curves may be applied directly to the input image. If additional control is needed, each sub-region may again be partitioned and local tone curves determined for each of the new sub-regions.

Multi-scale tone mapping

Systems, methods, and computer readable media for the use of a metric whose value is especially sensitive to the information lost when an image's pixels are clipped are disclosed. The metric may be used as an image's score, where higher values are indicative of lost highlight information (more clipped pixels). One use of the disclosed metric would be to determine when the use of high dynamic range (HDR) techniques are appropriate. The disclosed metric may also be used to bias a scene's exposure value (EV) such as to a lower or underexposed value (EV&#x2212;) so that the scene may be captured with no more than an acceptable number of clipped pixels.

Highlight exposure metric and its applications

Systems, methods, and computer readable media to capture and process high dynamic range (HDR) images when appropriate for a scene are disclosed. When appropriate, multiple images at a single&#x2014;slightly underexposed&#x2014;exposure value are captured (making a constant bracket HDR capture sequence) and local tone mapping (LTM) applied to each image. Local tone map and histogram information can be used to generate a noise-amplification mask which can be used during fusion operations. Images obtained and fused in the disclosed manner provide high dynamic range with improved noise and de-ghosting characteristics.

Constant bracket high dynamic range (cHDR) operations

A solid state imaging device includes an array of active pixels and an infrared cut filter formed over the sensor. Optionally, a slot in the infrared cut filter allows infrared illumination to reach the sensor to be detected by pixels covered by a visually opaque filter and surrounded by pixels of special types that limit charge leakage and enable high dynamic range sensing of infrared illumination. A ratio of average infrared signal to average brightness indicates an amount of infrared illumination reaching the imaging device.

Growing community of inventors

Mountain View, CA, United States of America

Xuemei Zhang