Apple Inc.

Gregory Guyomarc'h

Abhishek Dhanda

Mark N Gamadia

Moshe Laifenfeld

Yingjun Bai

Vignesh Jagadeesh

Andrew David Fernandez

Guillaume Tartavel

Xuemei Zhang

Yonghui Zhao

Wei Sun

Frederic Cao

Oriel Bergig

Michael Chin

Ling Xiao Wang

Jason R Holloway

Philip G Lee

Connor Henley

Alexandra H Chau

Kosta Dabov

Imaging apparatus includes a camera, which includes an autofocus mechanism. A range sensor includes a transmitter, which includes multiple emitters configured to transmit optical pulses at different angles toward the scene, and a receiver, to receive and sense respective times of flight of the optical pulses that are reflected from the scene. A photodetector outputs a signal indicative of an intensity of reflections from the scene. Control circuitry drives the emitters in alternation to transmit the optical pulses, estimates a range to a target in the scene responsively to the times of flight sensed by the receiver, detects a difference in the signal output by the photodetector in response to the pulses transmitted by different ones of the emitters, and sets the autofocus mechanism responsively to the estimated range as long as the detected difference is no greater than a predefined threshold difference.

Detection of close-range specular targets

Various embodiments disclosed herein include techniques for operating a multiple camera system. In some embodiments, a primary camera may be selected from a plurality of cameras using object distance estimates, distance error information, and minimum object distances for some or all of the plurality of cameras. In other embodiments, a camera may be configured to use defocus information to obtain an object distance estimate to a target object closer than a minimum object distance of the camera. This object distance estimate may be used to assist in focusing another camera of the multi-camera system.

Object depth estimation and camera focusing techniques for multiple-camera systems

Various embodiments disclosed herein include techniques for determining autofocus for a camera on a mobile device. In some instances, depth imaging is used to assist in determining a focus position for the camera through an autofocus process. For example, a determination of depth may be used to determine a focus position for the camera. In another example, the determination of depth may be used to assist another autofocus process.

Camera autofocus using time-of-flight assistance

Various methods for utilizing a saliency heatmaps are described. The methods include obtaining image data corresponding to an image of a scene, obtaining a saliency heatmap for the image of the scene based on a saliency network, wherein the saliency heatmap indicates a likelihood of saliency for a corresponding portion of the scene, and manipulating the image data based on the saliency heatmap. In embodiments, the saliency heatmap may be produced using a trained machine learning model. The saliency heatmap may be used for various image processing tasks, such as determining which portion(s) of a scene to base an image capture device's autofocus, auto exposure, and/or white balance operations upon. According to some embodiments, one or more bounding boxes may be generated based on the saliency heatmap, e.g., using an optimization operation, which bounding box(es) may be used to assist or enhance the performance of various image processing tasks.

Saliency of an object for image processing operations

Various embodiments include synchronization of camera focus movement control with frame capture. In some embodiments, such synchronization may comprise synchronized focus movement control that is based at least in part on integration timing and/or region of interest (ROI) timing. According to some examples, an actuator of a camera module may be controlled such that a lens group and/or an image sensor of the camera module move towards a focus position during one or more time periods (e.g., a non-integration time period in which the image sensor is not being exposed, a non-ROI time period in which a ROI of the image sensor is not being exposed for image capture, and/or a blanking interval, etc.). Additionally, or alternatively, the actuator may be controlled such that the lens group and the image sensor do not move relative to each other in a focus direction during one or more other time periods (e.g., an integration time period in which the image sensor is being exposed, a ROI time period in which the ROI of the image sensor is being exposed, etc.).

Synchronization of camera focus movement control with frame capture

This disclosure relates to the lightweight and efficient synthesis of shallow depth of field (SDOF) renderings. According to some embodiments, coarse focus information and semantic segmentation information may be leveraged to generate SDOF renderings in a 'live preview' or 'streaming' mode. Semantic segmentation may be defined as a process of creating a mask over an image, wherein pixels are segmented into a predefined set of semantic classes. Segmentations may include as many classes as are desired by a given implementation (e.g., a 'foreground' class and a 'background' class). In some embodiments, the rendering of synthetic SDOF images according to the techniques described herein may be completed using only a single camera of a device and without the use of dedicated depth-sensing technology (e.g., structured light cameras, stereo cameras, time-of-flight cameras, etc.), thereby allowing the SDOF rendering process to operate in a fashion that is not unduly time-, processing-, and/or power-intensive.

Shallow depth of field (SDOF) rendering

Techniques for auto white balancing of captured images based on detection of flicker in ambient light is described. When flicker is detected in ambient light during an image capture event, and the flicker is unchanging during the image capture event, a white point of image data may be estimated according to a first technique. When flicker is detected in ambient light during an image capture event, and the flicker is changing during the image capture event, a white point of image data may be estimated according to a second technique. When flicker is not detected, a white point of image data may be estimated according to a third technique. Image data may be color corrected based on the estimated white point.

Auto white balance control algorithm based upon flicker frequency detection

Systems, methods, and computer readable media to identify the operating characteristics of a multi-LED flash module for use with a image capture device are described. Flash modules having multiple LEDs have been found to exhibit large module-to-module variation in the light spectrum they emit. It has been determined that such variations make defining a 'golden module' impractical. To overcome this and other issues, techniques to characterize the operational behavior of a flash module having multiple LEDs using easily obtained measurements are disclosed. Such characterizations permit the characterization of each flash module to account for the particular lighting capability of its specific LEDs during image capture operations.

Growing community of inventors

San Francisco, CA, United States of America

Gregory Guyomarc'h