Apple Inc.

Yashas Rai Kurlethimar

Andrew B Watson

Jiaying Wu

Can Jin

Seyedkoosha Mirhosseini

Nicolas Pierre Marie Frederic Bonnier

Nicolas P Bonnier

Nathaniel C Begeman

Michael J Rockwell

Giovanni Carbone

Mehmet N Agaoglu

William W Sprague

Elijah H Kleeman

Alexander G Berardino

Milad Akhlaghi Bouzan

Jad-Nicolas Khoury

Prior to rendering a current frame, a device obtains a previously rendered frame. The device determines that a first portion of the previously rendered frame is associated with a particular type of content. The device renders a first portion of the current frame that corresponds to the first portion of the previously rendered frame with a first rendering characteristic while rendering a second portion of the current frame with a second rendering characteristic that is different from the first rendering characteristic.

Content based foveation

A vertical lookup table (LUT) stores first vertical scale values for a first display in a first portion and second vertical scale values for a second display in a second portion. A horizontal LUT stores horizontal scale values for the first display and the second display. A first version of visual content is generated for the first display by applying the first vertical scale values and the horizontal scale values to a first image frame that represents the visual content at a first resolution. A second version of the visual content is generated for the second display by applying the second vertical scale values and the horizontal scale values to a second image frame that represents the visual content at a second resolution.

Generating different versions of visual content for display

In one implementation, a method of generating an output image is performed by a device including a display, one or more processors, and non-transitory memory. The method includes obtaining a currently rendered image. The method includes obtaining an accumulation image based on previously rendered images. The method includes generating an output image based on the currently rendered image and the accumulation image, wherein the output image includes: (1) a first output pixel at a first pixel location associated with a first resolution greater than or equal to a threshold, the first output pixel having a first output pixel value that is a currently rendered image pixel value of a currently rendered image pixel at the first pixel location; (2) a second output pixel at a second pixel location associated with a second resolution less than the threshold, the second output pixel having a second output pixel value that is a weighted sum of a currently rendered image pixel value of a currently rendered image pixel at the second pixel location and an accumulation image pixel value of an accumulation image pixel at a corresponding pixel location corresponding to the second pixel location; and (3) a third output pixel at a third pixel location associated with a third resolution less than the threshold, the third output pixel having a third output pixel value that is a currently rendered image pixel value of a currently rendered image pixel at the third pixel location. The method includes displaying, on the display, the output image.

Foveated anti-aliasing

An electronic device that includes a display and an eye tracker configured to collect eye tracking data regarding a gaze of one or more of a user's eyes across the display is disclosed herein. The electronic device includes processing circuitry that is operatively coupled to the display and configured to foveate one or more areas of the display according to the eye tracking data. If the eye tracking data input is lost, the processing circuitry is configured to recover from the loss of eye tracking data by changing one or more aspects of the foveated areas (e.g., size, resolution, etc.) until a threshold is satisfied. As time elapses since loss of eye tracking, the foveated areas move toward a center or a salient region of the display.

Recovery from eye-tracking loss in foveated displays

A head-mounted display may display images in eye boxes. Right and left gaze tracking sensors may gather left eye gaze information from a left eye box and right eye gaze information from a right eye box. During operation in a first mode in which the right and left gaze tracking sensors are not impaired, gaze information from the right and left gaze tracking sensors may be used in presenting foveated and distortion-corrected image content on the displays and/or in otherwise operating the left and right displays. During operation in a second mode in which one of the gaze tracking sensors is impaired, gaze information for the impaired sensor may be estimated from gaze information for the unimpaired sensor in combination with an object depth estimate based on depth buffer information, vestibulo-ocular-reflex gain information, and historical information.

Electronic devices with monocular gaze estimation capabilities

An electronic device may have a foveated display, an eye-tracking system and a head movement detection system. The eye-tracking system may gather information on a user's point of regard on the display while the head movement detection system may capture information regarding the rotation of the observer's head. Based on the point-of-regard information, head rotation information, image data, the type of eye/head movement that is underway, and/or tiredness information, control circuitry in the electronic device may produce image data for a display, with areas of different resolutions and(or) visual quality. A full-resolution and(or) quality portion of the image may overlap the point of regard. One or more lower resolution portions of the image may surround the full-resolution portion. The control circuitry may include a gaze prediction system for predicting the movement of the user's gaze during a saccade.

Growing community of inventors

Sunnyvale, CA, United States of America

Yashas Rai Kurlethimar