Apple Inc.

Nicolas P Bonnier

Jiaying Wu

Teun R Baar

Ramin Samadani

Marc Albrecht

Christina G Gambacorta

Elijah H Kleeman

Cheng Kai Chen

Moinul H Khan

Hao Pan

Paul M Hubel

Tobias Eble

Roy J E M Raymann

Ye Cong

John Z Zhong

Wei Chen

Jim Chen Chou

Mahesh B Chappalli

Arthur Yasheng Zhang

Noah Denis Bedard

Lu Zhang

Paul V Johnson

Adria Fores Herranz

Tobias Jung

Alexandre Chapiro

Can Jin

Bosheng Zhang

Yashas Rai Kurlethimar

Jonathan C Moisant-Thompson

Cody J White

Randall R Rauwendaal

William Mathews Riedel

Paul Stephen Drzaic

Hung Sheng Lin

Hyunwoo Nho

Ian C Hendry

Martin Paul Grunthaner

Jie Won Ryu

Sun-il Chang

Garrett M Johnson

Farhan A Baqai

Alexandre Da Veiga

Rui Zhang

Junhua Tan

Graham D Finlayson

Graham Butler Myhre

Bruno M Sommer

ByoungSuk Kim

Nathaniel C Begeman

David Andrew Doyle

Yifan Zhang

Andrew B Watson

David S Zalatimo

Sean C Chang

Angelo M Alaimo

Ilya Romanenko

Giovanni Carbone

Tae-Wook Koh

Bennett S Wilburn

Kathrin Berkner Cieslicki

Jeffrey S Norris

Ove Lyngnes

Mehmet N Agaoglu

Seyed Hesameddin Najafi Shoushtari

William W Sprague

Gregory B Abbas

Bryan Dang

Jonathan T Varbel

Claus Moelgaard

Alexander G Berardino

Zuo Xia

Sebastian Knitter

Seth E Hunter

Guillaume Lestoquoy

Jackson K Roland

Jason P De Villiers

Tongbo Chen

Milad Akhlaghi Bouzan

Alexandra S Ludlow

Tobias Baldauf

Hau Ling Hung

Victor M De Araujo Oliveira

Embodiments disclosed herein are directed to devices and methods for performing spatially-varying brightness correction of a flash image. The spatially-varying brightness correction utilizes a set of images that includes one or more images captured under flash illumination and one or more images captured without flash illumination, which are used to decompose one of the flash images or an image generated therefrom into a flash contribution image and an ambient contribution image. The flash contribution image and the ambient contribution image are recombined to generate a corrected flash image, and information about the flash module and/or scene is used to adjust the relative contributions of these images during recombination.

Techniques for correcting images in flash photography

A light source module includes an array of illumination elements and an optional projecting lens. The light source module is configured to receive or generate a control signal for adjusting different ones of the illumination elements to control a light field emitted from the light source module. In some embodiments, the light source module is also configured to adjust the projecting lens responsive to objects in an illuminated scene and a field of view of an imaging device. A controller for a light source module may determine a light field pattern based on various parameters including a field of view of an imaging device, an illumination sensitivity model of the imaging device, depth, ambient illumination and reflectivity of objects, configured illumination priorities including ambient preservation, background illumination and direct/indirect lighting balance, and so forth.

Adaptive-flash photography, videography, and/or flashlight using camera, scene, or user input parameters

An electronic device may have a display and a camera. Control circuitry in the device can gather information on a user's point of gaze using a gaze tracking system and other sensors, can gather information on the real-world image such as information on content, motion, and other image attributes by analyzing the real-world image, can gather user vision information such as user acuity, contrast sensitivity, field of view, and geometrical distortions, can gather user input such as user preferences and user mode selection commands, and can gather other input. Based on the point-of-gaze information and/or other gathered information, the control circuitry can display the real-world image and supplemental information on the display. The supplemental information can include augmentations such as icons, text labels, and other computer-generated text and graphics overlaid on the real world image and can include enhanced image content such as magnified portions of the real-world image.

Image enhancement devices with gaze tracking

In one implementation, a method includes receiving a warped image representing simulated reality (SR) content (e.g., to be displayed in a display space), the warped image having a plurality of pixels at respective locations uniformly spaced in a grid pattern in a warped space, wherein the plurality of pixels are respectively associated with a plurality of respective pixel values and a plurality of respective scaling factors indicating a plurality of respective resolutions at a plurality of respective locations of the SR content (e.g., in the display space). The method includes processing the warped image in the warped space based on the plurality of respective scaling factors to generate a processed warped image and transmitting the processed warped image.

Dynamic foveated pipeline

A method includes obtaining first pass-through image data characterized by a first pose. The method includes obtaining respective pixel characterization vectors for pixels in the first pass-through image data. The method includes identifying a feature of an object within the first pass-through image data in accordance with a determination that pixel characterization vectors for the feature satisfy a feature confidence threshold. The method includes displaying the first pass-through image data and an AR display marker that corresponds to the feature. The method includes obtaining second pass-through image data characterized by a second pose. The method includes transforming the AR display marker to a position associated with the second pose in order to track the feature. The method includes displaying the second pass-through image data and maintaining display of the AR display marker that corresponds to the feature of the object based on the transformation.

Methods and devices for detecting and identifying features in an AR/VR scene

Eyewear such as sunglasses may include display-optimized lenses. The lenses may be optimized for viewing an external display and/or for viewing a display in the head-mounted device while also providing sun protection for the user's eyes. The lenses may include a polarizer and a color filter that are designed for a given target display. Lenses that are optimized for a display that emits linearly polarized light may include a linear polarizer. Lenses that are optimized for a display that emits circularly polarized light may include a circular polarizer. The circular polarizer may include a quarter wave plate and a linear polarizer. The color filter may have a transmission spectrum curve with peaks corresponding to the primary colors of the target display so that the color and brightness of display light is preserved while the brightness of sunlight is reduced.

Eyewear with display-optimized lenses

An electronic device such as a watch may include a display and a light sensor located behind the display. The light sensor may be used to measure the color of external objects. During color sampling operations, the display may emit light towards the external object in front of the display while the light sensor gathers color measurements. The display may emit light of different colors and the light sensor may detect an amount of reflected light for each color, which in turn may be used to determine the color of the external object. The control circuitry may use a watch-band-specific algorithm to determine the color of watch bands and may use a clothing-specific algorithm to determine the color of clothing. The control circuitry may display the color on the display so that the face of the watch matches the user's clothing or matches the user's watch band.

Electronic devices with color sampling sensors

A head-mounted display may display images in eye boxes. Right and left gaze tracking sensors may gather left eye gaze information from a left eye box and right eye gaze information from a right eye box. During operation in a first mode in which the right and left gaze tracking sensors are not impaired, gaze information from the right and left gaze tracking sensors may be used in presenting foveated and distortion-corrected image content on the displays and/or in otherwise operating the left and right displays. During operation in a second mode in which one of the gaze tracking sensors is impaired, gaze information for the impaired sensor may be estimated from gaze information for the unimpaired sensor in combination with an object depth estimate based on depth buffer information, vestibulo-ocular-reflex gain information, and historical information.

Electronic devices with monocular gaze estimation capabilities

Systems, methods, and devices are provided to reduce a likelihood of image burn-in on an electronic display. Such an electronic device may include image processing circuitry and an electronic display. The image processing circuitry may receive image data and analyze the image data for risk of image burn-in and, based at least in part on the analysis of the image data, reduce a risk of image burn-in at least in part by reducing a local maximum pixel luminance value in at least one of a plurality of regions of the image data over time or by reducing a dynamic range headroom of the image data. The electronic display may display the image data with a reduced risk of image burn-in on the pixels of the electronic display.

Electronic display burn-in detection and mitigation

Aspects of the subject technology relate to display circuitry for displaying both standard dynamic range (SDR) and high dynamic range (HDR) content with an HDR display. The subject technology provides a headroom-based transfer function that maintains the contrast of the SDR content whether the display is operated in the SDR or HDR mode.

Standard and high dynamic range display systems and methods for high dynamic range displays

An electronic device may include a display that displays virtual reality content. Control circuitry may estimate a brightness adaptation state of a user that is wearing the electronic device. The control circuitry may select a tone mapping curve and brightness level for the virtual reality content based on the user's adaptation state. To estimate the user's adaptation state, the control circuitry may gather ambient light information from an ambient light sensor, may gather physiological attributes of the user such as blink rate, pupil size, and eye openness from a camera, and may gather gaze position information from gaze detection circuitry. The control circuitry may optimize the brightness of the display based on the user's current adaptation state, or the control circuitry may shift the brightness of the display away from the user's adaptation level to help guide the adaptation state to the desired level.

Electronic device with adaptive display

An electronic device may be provided with a display. A content generator such as a camera may capture images in high dynamic range mode or standard dynamic range mode. The images may have associated image metadata such as face detection information, camera settings, color and luminance histograms, and image classification information. Control circuitry in the electronic device may determine tone mapping parameters for the captured images based on the image metadata. The tone mapping parameters for a given image may be stored with the image in the metadata file. When it is desired to display the image, the control circuitry may apply a tone mapping process to the image according to the stored tone mapping parameters. The algorithm that is used to determine tone mapping parameters based on image metadata may be based on user preference data gathered from a population of users.

Tone mapping techniques for increased dynamic range

A head-mounted device may include a display that generates content and an optical system through which the content is viewable. The head-mounted device may include a lighting system that illuminates a periphery of the optical system. When the user places the device on his or her head in a brightly lit environment, control circuitry may operate the lighting system to provide bright illumination to the user's peripheral vision. The lighting system may gradually decrease in brightness until the user transitions from a bright-adapted state to a dark-adapted state. When the user is partially or fully dark-adapted, the lighting system may be turned off and the display may be turned on. In some arrangements, an ambient light sensor may measure ambient light conditions outside of the electronic device and the control circuitry may control the lighting system based on the ambient lighting conditions.

Electronic device with adaptive lighting system

Growing community of inventors

Campbell, CA, United States of America

Nicolas P Bonnier