Apple Inc.

Simon Fortin-Deschenes

Anthony Ghannoum

Vincent Chapdelaine-Couture

Yan Cote

Moinul H Khan

James H Zhou

Bertrand Nepveu

Bennett S Wilburn

Nan Zhu

Marc-Andre Chenier

Sterling G Orsten

Yanli Zhang

Ian Powell

Irving Lustigman

A Head-Mounted Display system together with associated techniques for performing accurate and automatic inside-out positional, user body and environment tracking for virtual or mixed reality are disclosed. The system uses computer vision methods and data fusion from multiple sensors to achieve real-time tracking. High frame rate and low latency is achieved by performing part of the processing on the HMD itself.

Head-mounted display for virtual and mixed reality with inside-out positional, user body and environment tracking

A Head-Mounted Display with camera sensors to perform chroma keying in a mixed reality context is presented. Low latency is achieved by embedding the processing in the HMD itself, specifically, format camera images, detect the selected color range and make a composite with the virtual content.

Low latency chroma keying embedded in a head-mounted display for mixed reality

An optical lens arrangement comprises a first Fresnel lens element and a second lens element. The first Fresnel lens element defines a flat surface side and an opposite faceted surface side defining wedge and draft faces. The flat surface side faces towards the eye of a user and the opposite faceted surface side faces away from the eye of the user. The second lens element interfaces with the faceted surface side of first Fresnel lens. The second lens element is selected from the group consisting of: a second Fresnel lens element, a singlet lens element, a doublet lens element and any combination thereof. The first Fresnel lens is proximal relative to the eye of the user and the second lens element is distal relative to the eye of the user. Head mounted devices (HMD) including these optical lens arrangements are provided. Methods of making such optical lens arrangements and HMDs are also provided.

Optical arrangements including Fresnel lens elements

Some implementations provide improved user experiences on head mounted devices (HMDs) that provide near eye viewing, e.g., HMDs that display distorted images and provide lenses that undistort the images for the user. The images are produced using distortion that is corrected dynamically based on context to conserve device resources. To do so, a context associated with a state of the user, the HMD, or content being viewed on the HMD is tracked during the user experience. For example, the device may predict pupil position, eye state, eye gaze direction, or eye fixation, content type, connection mode, and other context. The device uses the tracked context to determine how to correct distortion for the images at different points during the user experience. For example, new distortion corrections may be computed and used while the user's gaze is moving and previously-determined distortion corrections may be used while the user's gaze is fixed.

Context aware dynamic distortion correction

Growing community of inventors

Sunnyvale, CA, United States of America

Simon Fortin-Deschenes