Apple Inc.

United States of America As Represented by the Administrator of Nasa

The United States of America As Represented by the Administrator of the

Other

Andrew B Watson

Mehmet N Agaoglu

Jiaying Wu

Yashas Rai Kurlethimar

Nicolas Pierre Marie Frederic Bonnier

Can Jin

Albert J Ahumada, Jr

Chaohao Wang

Yang Li

Nicolas P Bonnier

Alexandre Chapiro

Yi-Pai Huang

Elijah H Kleeman

Alexander G Berardino

Pretesh A Mascarenhas

Laura Lynn Walker

Tim H W Cornelissen

A vertical lookup table (LUT) stores first vertical scale values for a first display in a first portion and second vertical scale values for a second display in a second portion. A horizontal LUT stores horizontal scale values for the first display and the second display. A first version of visual content is generated for the first display by applying the first vertical scale values and the horizontal scale values to a first image frame that represents the visual content at a first resolution. A second version of the visual content is generated for the second display by applying the second vertical scale values and the horizontal scale values to a second image frame that represents the visual content at a second resolution.

Generating different versions of visual content for display

An electronic device may have a display and a gaze-tracking system. The display may use different resolutions in different portions of the display. Point-of-gaze information from the gaze-tracking system may be used to align a high-resolution portion of the display with the viewer's point-of-gaze. In the surrounding portions of the display, the resolution of the display may be decreased. This reduces power consumption, computation power, and bandwidth requirements without sacrificing the user experience. The resolution of the human eye is lower for purely chromatic signals than it is for luminance (brightness signals). To exploit this, the image data for the display may have an achromatic (white/black) luminance channel and two chromatic opponent channels. The opponent channels may then be compressed by a greater amount than the luminance channel, which increases the overall compression of the image data.

Display with chromatic foveation

Tracking an eye characteristic (e.g., gaze direction or pupil position) of a user's eyes by staggering image capture and using a predicted relationship between the user's eyes between eye captures to predict that eye's eye characteristic between those eye captures. Images of a user's eyes are captured in a staggered manner in the sense that the images of second eye are captured between the capture times of the images of the first eye and vice versa. An eye characteristic of the first eye at the capture times is determined based on the images of the first eye at those times. In addition, the eye characteristic of that first eye is predicted at additional times between captures based on a predicted relationship between the eyes.

Eye tracking using efficient image capture and vergence and inter-pupillary distance history

An electronic device may have a display for displaying image content. Head-mounted support structures in the device may be used to support the display. The electronic device may have an eye monitoring system that detects eye saccades and eye blinks. Control circuitry in the electronic device may coordinate operation of the display with periods of suppressed visual sensitivity that are associated with the saccades and blinks. By making adjustments to display circuitry and image content during periods of suppressed visual sensitivity, potentially visually obtrusive changes to displayed images can be hidden from a user of the electronic device. Adjustments to display operation may help reduce burn-in effects, may help reduce power consumption, and may otherwise improve device performance.

Electronic devices with display operation based on eye activity

An electronic device that includes a display and an eye tracker configured to collect eye tracking data regarding a gaze of one or more of a user's eyes across the display is disclosed herein. The electronic device includes processing circuitry that is operatively coupled to the display and configured to foveate one or more areas of the display according to the eye tracking data. If the eye tracking data input is lost, the processing circuitry is configured to recover from the loss of eye tracking data by changing one or more aspects of the foveated areas (e.g., size, resolution, etc.) until a threshold is satisfied. As time elapses since loss of eye tracking, the foveated areas move toward a center or a salient region of the display.

Recovery from eye-tracking loss in foveated displays

In an embodiment, an electronic device includes a display and an eye tracker. The display includes one or more foveated areas. In the embodiment, the eye tracker is configured to collect eye tracking data regarding a gaze of one or more eyes of a user on the display. The electronic device also includes processing circuitry operatively coupled to the display. In the embodiment, the processing circuitry is configured to receive an indication of a motion associated with the gaze from the eye tracker. The processing circuitry is also configured to determine a previous location associated with the gaze during a previous frame and a target position associated with the gaze during a target frame. In the embodiment, the processing circuitry is configured to expand one or more foveated areas of the display adjacent a previous position of the gaze of the user.

Peripheral luminance or color remapping for power saving

Techniques are provided for implementing near-view notification techniques. In some instances, a viewing distance with respect to a display screen of an electronic device may be determined. In accordance with a determination that the viewing distance is above a distance threshold, content may be presented on the display screen. In accordance with a determination that the viewing distance is under a distance threshold, until the viewing distance returns to being above the distance threshold, a visual notification may be displayed. The visual notification may disrupt the content presented on the display screen.

Near-viewing notification techniques

An electronic device may have a foveated display, an eye-tracking system and a head movement detection system. The eye-tracking system may gather information on a user's point of regard on the display while the head movement detection system may capture information regarding the rotation of the observer's head. Based on the point-of-regard information, head rotation information, image data, the type of eye/head movement that is underway, and/or tiredness information, control circuitry in the electronic device may produce image data for a display, with areas of different resolutions and(or) visual quality. A full-resolution and(or) quality portion of the image may overlap the point of regard. One or more lower resolution portions of the image may surround the full-resolution portion. The control circuitry may include a gaze prediction system for predicting the movement of the user's gaze during a saccade.

Electronic device with foveated display and gaze prediction

A method of measuring the video acuity of a physical imaging system is disclosed. A chart comprising an image of a set of test symbols is provided. The set of test symbols comprises a plurality of symbols repeated at a plurality of symbol sizes. A digital image of the chart is obtained using the imaging system. The digital image is cropped to obtain a set of cropped images comprising individual digital images of each test symbol in the set of test symbols. A template image of a test symbol is aligned with a test image comprising one member of the set of cropped images. The template image comprises a digitally generated image of the test symbol in the test image. The normalized correlation between the aligned template image and the test image is determined. The test symbol whose template image has the highest correlation with the test image is identified.

Measuring video acuity

A method of measuring motion blur is disclosed comprising obtaining a moving edge temporal profile r(k) of an image of a high-contrast moving edge, calculating the masked local contrast m(k) for r(k) and the masked local contrast m(k) for an ideal step edge waveform r(k) with the same amplitude as r(k), and calculating the measure or motion blur &#x3a8; as a difference function,&#x3a9;=(&#x394;()&#x2212;()|).The masked local contrasts are calculated using a set of convolution kernels scaled to simulate the performance of the human visual system, and &#x3a8; is measured in units of just-noticeable differences.

Visible motion blur

A method for generating a visual acuity metric, based on wavefront aberrations (WFAs), associated with a test subject and representing classes of imperfections, such as defocus, astigmatism, coma and spherical aberrations, of the subject's visual system. The metric allows choices of different image template, can predict acuity for different target probabilities, can incorporate different and possibly subject-specific neural transfer functions, can predict acuity for different subject templates, and incorporates a model of the optotype identification task.

Growing community of inventors

Los Gatos, CA, United States of America

Andrew B Watson