Apple Inc.

Mehmet N Agaoglu

Andrew B Watson

Alexander G Berardino

Cheng Kai Chen

Chaohao Wang

Zhibing Ge

Jiaying Wu

Yang Li

Shih-Chyuan Fan Jiang

Nicolas P Bonnier

Gokhan Avkarogullari

Yingying Tang

Jenny Hu

Julian K Shutzberg

James R Montgomerie

Alexandre Chapiro

Alexander K Kan

Harsha Shirahatti

Yi-Pai Huang

Yashas Rai Kurlethimar

Leah M Gum

Paul A Lacey

William W Sprague

Nicolas Pierre Marie Frederic Bonnier

Elijah H Kleeman

Nischay Goel

Wanqing Xin

Andrey Pokrovskiy

Vinay Chawda

Pretesh A Mascarenhas

Tim H Cornelissen

Yuhui Li

Tracking an eye characteristic (e.g., gaze direction or pupil position) of a user's eyes by staggering image capture and using a predicted relationship between the user's eyes between eye captures to predict that eye's eye characteristic between those eye captures. Images of a user's eyes are captured in a staggered manner in the sense that the images of second eye are captured between the capture times of the images of the first eye and vice versa. An eye characteristic of the first eye at the capture times is determined based on the images of the first eye at those times. In addition, the eye characteristic of that first eye is predicted at additional times between captures based on a predicted relationship between the eyes.

Eye tracking using efficient image capture and vergence and inter-pupillary distance history

An electronic device may have a display for displaying image content. Head-mounted support structures in the device may be used to support the display. The electronic device may have an eye monitoring system that detects eye saccades and eye blinks. Control circuitry in the electronic device may coordinate operation of the display with periods of suppressed visual sensitivity that are associated with the saccades and blinks. By making adjustments to display circuitry and image content during periods of suppressed visual sensitivity, potentially visually obtrusive changes to displayed images can be hidden from a user of the electronic device. Adjustments to display operation may help reduce burn-in effects, may help reduce power consumption, and may otherwise improve device performance.

Electronic devices with display operation based on eye activity

An electronic device may include a display. Control circuitry may operate the display at different frame rates such as 60 Hz, 80 Hz, and 120 Hz. The control circuitry may determine which frame rate to use based on a speed of animation on the display and based on a type of animation on the display. To mitigate the appearance of judder as the display frame rate changes, the control circuitry may implement techniques such as hysteresis (e.g., windows of tolerance around speed thresholds to ensure that the display frame rate does not change too frequently as a result of noise), speed thresholds that are based on a user perception study, consistent latency between touch input detection and corresponding display output across different frame rates (e.g., using a fixed touch scan rate that is independent of frame duration), and animation-specific speed thresholds for triggering frame rate changes.

Displays with variable frame rates

Various implementations provide views of 3D environments (e.g., extended reality (XR) environments). Non-eye-based user activity, such as hand gestures, is associated with some types of eye-based activity, such as the user gazing at a particular user interface component displayed within a view of a 3D environment. For example, a user's pinching hand gesture may be associated with the user gazing at a particular user interface component, such as a button, at around the same time as the pinching hand gesture is made. These associated behaviors (e.g., the pinch and gaze at the button) may then be interpreted as user input, e.g., user input selecting or otherwise acting upon that user interface component. In some implementations, non-eye-based user activity is only associated with types of eye-based user activity that are likely to correspond to a user perceiving what they are seeing and/or intentionally looking at something.

User interface response based on gaze-holding event assessment

In an embodiment, an electronic device includes a display and an eye tracker. The display includes one or more foveated areas. In the embodiment, the eye tracker is configured to collect eye tracking data regarding a gaze of one or more eyes of a user on the display. The electronic device also includes processing circuitry operatively coupled to the display. In the embodiment, the processing circuitry is configured to receive an indication of a motion associated with the gaze from the eye tracker. The processing circuitry is also configured to determine a previous location associated with the gaze during a previous frame and a target position associated with the gaze during a target frame. In the embodiment, the processing circuitry is configured to expand one or more foveated areas of the display adjacent a previous position of the gaze of the user.

Peripheral luminance or color remapping for power saving

Aspects of the subject technology relate to gaze-dependent visual encryption of electronic device displays. Each display frame that is displayed on the electronic device display may include a clear-display region around the user's gaze location and an obscured region outside the clear-display region. In this way, only the display content that the user is actively viewing is recognizable and understandable and an onlooker such as an unwanted observer looking over the user's shoulder is unable to understand what is displayed. The obscured region of each display frame may be generated such that the overall look and structure of that region is unchanged, but the content is unintelligible. In this way, the visual experience of the user is not disrupted or distracted by the visual encryption and the eye of the onlooker is not guided to the clear-display region by the visual encryption.

Gaze-dependent display encryption

An electronic device may have a foveated display, an eye-tracking system and a head movement detection system. The eye-tracking system may gather information on a user's point of regard on the display while the head movement detection system may capture information regarding the rotation of the observer's head. Based on the point-of-regard information, head rotation information, image data, the type of eye/head movement that is underway, and/or tiredness information, control circuitry in the electronic device may produce image data for a display, with areas of different resolutions and(or) visual quality. A full-resolution and(or) quality portion of the image may overlap the point of regard. One or more lower resolution portions of the image may surround the full-resolution portion. The control circuitry may include a gaze prediction system for predicting the movement of the user's gaze during a saccade.

Electronic device with foveated display and gaze prediction

Growing community of inventors

Santa Clara, CA, United States of America

Mehmet N Agaoglu