Magic Leap, Inc.

Samuel A Miller

Lionel Ernest Edwin

Ivan Li Chuen Yeoh

Rony Abovitz

Brian T Schowengerdt

Paul M Greco

William Hudson Welch

George Alistair Sanger

Lomesh Agarwal

Zachary C Nienstedt

Edwin Joseph Selker

Yonatan Munk

David Charles Lundmark

Jehangir Tajik

Randall E Hand

Brian Lloyd Schmidt

Anastasia Andreyevna Tajik

Ivan L Yeoh

Jeffrey Scott Sommers

Bradley Vincent Stuart

Daniel Farmer

Philip O'Connor

Sergey Fyodorovich Prokushkin

Gary R Bradski

Yan Xu

Jordan Alexander Cazamias

Nicholas Atkinson Kramer

Eric C Browy

Barak Freedman

Robert Blake Taylor

Michael Janusz Woods

Ashwin Swaminathan

Ali Shahrokni

Anush Mohan

Eran Guendelman

Xuan Zhao

Prateek Singhal

Rafael Domingos Torres

Daniel Olshansky

Bjorn Nicolaas Servatius Vlaskamp

Christian Ivan Robert Moore

Joel David Holder

Hélder Toshiro Suzuki

Siddharth Choudhary

Graeme John Devine

Jeremy Dwayne Miranda

Ron Liraz Lidji

Wenyi Zhao

Adam Charles Carlson

Etienne Gregoire Grossmann

Michael Robert Johnson

Erik Fonseka

Shiuli Das

Suraj Manjunath Shanbhag

David Thomas Roach

Tushar Arora

Paul Lacey

Nukul Sanjay Shah

Po-Kang Huang

Silas Warren

Omar Khan

Hiral Honar Barot

Brian Christopher Clark

Brian Frederick Schmidt

Michael Harold Liebenow

Richmond B Chan

Terry Micheal O'Gara

Kenneth William Tossell

Lionel E Edwin

Brian Schwab

Martin Georg Zahnert

Dominik Michael Kasper

Aaron Mark Schuelke

Christopher Lopez

Brian Edward Oliver Bucknor

Manel Quim Sanchez Nicuesa

Spencer Lindsay

Terry Michael O'Gara

Aaron M Clarke

Nathan Yuki Baumli

Emily Elizabeth Mount

Nikola Stan

Andrea Lampart

Hirai Honar Barot

Rafa Gomez-Jordana Manas

Daniel Roberts

Philip O'connor

Ivan Li Yeoh

A display system includes a head-mounted display configured to project light, having different amounts of wavefront divergence, to an eye of a user to display virtual image content appearing to be disposed at different depth planes. The wavefront divergence may be changed in discrete steps, with the change in steps being triggered based upon whether the user is fixating on a particular depth plane. The display system may be calibrated for switching depth planes for a main user. Upon determining that a guest user is utilizing the system, rather than undergoing a full calibration, the display system may be configured to switch depth planes based on a rough determination of the virtual content that the user is looking at. The virtual content may have an associated depth plane and the display system may be configured to switch to the depth plane of that virtual content.

Depth plane selection for multi-depth plane display systems by user categorization

Systems and methods are disclosed for operating a head-mounted display system based on user perceptibility. The display system may be an augmented reality display system configured to provide virtual content on a plurality of depth planes by presenting the content with different amounts of wavefront divergence. Some embodiments include obtaining an image captured by an imaging device of the display system. Whether a threshold measure or more of motion blur is determined to be exhibited in one or more regions of the image. Based on a determination that the threshold measure or more of motion blur is exhibited in one or more regions of the image, one or more operating parameters of the wearable display are adjusted. Example operating parameter adjustments comprise adjusting the depth plane on which content is presented (e.g., by switching from a first depth plane to a second depth plane), adjusting a rendering quality, and adjusting power characteristics of the system.

Systems and methods for operating a display system based on user perceptibility

Wearable systems and method for operation thereof incorporating headset and controller localization using headset cameras and controller fiducials are disclosed. A wearable system may include a headset and a controller. The wearable system may alternate between performing headset tracking and performing controller tracking by repeatedly capturing images using a headset camera of the headset during headset tracking frames and controller tracking frames. The wearable system may cause the headset camera to capture a first exposure image an exposure above a threshold and cause the headset camera to capture a second exposure image having an exposure below the threshold. The wearable system may determine a fiducial interval during which fiducials of the controller are to flash at a fiducial frequency and a fiducial period. The wearable system may cause the fiducials to flash during the fiducial interval in accordance with the fiducial frequency and the fiducial period.

Wearable system with controller localization using headset cameras and controller fiducials

A display system may include a head-mounted display (HMD) for rendering a three-dimensional virtual object which appears to be located in an ambient environment of a user of the display. One or more eyes of the user may not be in desired positions, relative to the HMD, to receive, or register, image information outputted by the HMD and/or to view an external environment. For example, the HMD-to-eye alignment may vary for different users and/or may change over time (e.g., as the HMD is displaced). The display system may determine a relative position or alignment between the HMD and the user's eyes. Based on the relative positions, the wearable device may determine if it is properly fitted to the user, may provide feedback on the quality of the fit to the user, and/or may take actions to reduce or minimize effects of any misalignment.

Display systems and methods for determining registration between a display and a user's eyes

Examples of wearable systems and methods can use multiple inputs (e.g., gesture, head pose, eye gaze, voice, totem, and/or environmental factors (e.g., location)) to determine a command that should be executed and objects in the three-dimensional (3D) environment that should be operated on. The wearable system can detect when different inputs converge together, such as when a user seeks to select a virtual object using multiple inputs such as eye gaze, head pose, hand gesture, and totem input. Upon detecting an input convergence, the wearable system can perform a transmodal filtering scheme that leverages the converged inputs to assist in properly interpreting what command the user is providing or what object the user is targeting.

Identifying convergence of sensor data from first and second sensors within an augmented reality wearable device

One embodiment is directed to a user display device comprising a housing frame mountable on the head of the user, a lens mountable on the housing frame and a projection sub system coupled to the housing frame to determine a location of appearance of a display object in a field of view of the user based at least in part on at least one of a detection of a head movement of the user and a prediction of a head movement of the user, and to project the display object to the user based on the determined location of appearance of the display object.

Frame-by-frame rendering for augmented or virtual reality systems

A display subsystem for a virtual image generation system used by an end user, comprises first and second waveguide apparatuses, first and second projection subassemblies configured for introducing first and second light beams respectively into the first and second waveguide apparatuses, such that at least a first light ray and at least a second light ray respectively exit the first and second waveguide apparatuses to display first and second monocular images as a binocular image to the end user, and a light sensing assembly configured for detecting at least one parameter indicative of a mismatch between the displayed first and second monocular images as the binocular image.

Systems and methods for display binocular deformation compensation

Examples of a light field metrology system for use with a display are disclosed. The light field metrology may capture images of a projected light field, and determine focus depths (or lateral focus positions) for various regions of the light field using the captured images. The determined focus depths (or lateral positions) may then be compared with intended focus depths (or lateral positions), to quantify the imperfections of the display. Based on the measured imperfections, an appropriate error correction may be performed on the light field to correct for the measured imperfections. The display can be an optical display element in a head mounted display, for example, an optical display element capable of generating multiple depth planes or a light field display.

Light field display metrology

A wearable device may include a head-mounted display (HMD) for rendering a three-dimensional (3D) virtual object which appears to be located in an ambient environment of a user of the display. The relative positions of the HMD and one or more eyes of the user may not be in desired positions to receive, or register, image information outputted by the HMD. For example, the HMD-to-eye alignment may vary for different users and may change over time (e.g., as a user moves around and/or the HMD slips or is otherwise displaced). The wearable device may determine a relative position or alignment between the HMD and the user's eyes. Based on the relative positions, the wearable device may determine if it is properly fitted to the user, may provide feedback on the quality of the fit to the user, and may take actions to reduce or minimize effects of any misalignment.

The systems and methods described can include approaches to calibrate head-mounted displays for improved viewing experiences. Some methods include receiving data of a first target image associated with an undeformed state of a first eyepiece of a head-mounted display device; receiving data of a first captured image associated with deformed state of the first eyepiece of the head-mounted display device; determining a first transformation that maps the first captured image to the image; and applying the first transformation to a subsequent image for viewing on the first eyepiece of the head-mounted display device.

Compensation for deformation in head mounted display systems

A method of operating a virtual image generation system comprises allowing an end user to interact with a three-dimensional environment comprising at least one virtual object, presenting a stimulus to the end user in the context of the three-dimensional environment, sensing at least one biometric parameter of the end user in response to the presentation of the stimulus to the end user, generating biometric data for each of the sensed biometric parameter(s), determining if the end user is in at least one specific emotional state based on the biometric data for the each of the sensed biometric parameter(s), and performing an action discernible to the end user to facilitate a current objective at least partially based on if it is determined that the end user is in the specific emotional state(s).

Technique for controlling virtual image generation system using emotional states of user

A method for placing content in an augmented reality system. A notification is received regarding availability of new content to display in the augmented reality system. A confirmation is received that indicates acceptance of the new content. Three dimensional information that describes the physical environment is provided, to an external computing device, to enable the external computing device to be used for selecting an assigned location in the physical environment for the new content. Location information is received, from the external computing device, that indicates the assigned location. A display location on a display system of the augmented reality system at which to display the new content so that the new content appears to the user to be displayed as an overlay at the assigned location in the physical environment is determined, based on the location information. The new content is displayed on the display system at the display location.

Virtual location selection for virtual content

A display system can include a head-mounted display configured to project light to an eye of a user to display virtual image content at different amounts of divergence and collimation. The display system can include an inward-facing imaging system that images the user's eye and processing electronics that are in communication with the inward-facing imaging system and that are configured to obtain an estimate of a center of rotation of the user's eye. The display system may render virtual image content with a render camera positioned at a determined position relative to the determined position of the center of rotation of said eye.

Eye center of rotation determination, depth plane selection, and render camera positioning in display systems

One embodiment is directed to a system for enabling two or more users to interact within a virtual world comprising virtual world data, comprising a computer network comprising one or more computing devices, the one or more computing devices comprising memory, processing circuitry, and software stored at least in part in the memory and executable by the processing circuitry to process at least a portion of the virtual world data; wherein at least a first portion of the virtual world data originates from a first user virtual world local to a first user, and wherein the computer network is operable to transmit the first portion to a user device for presentation to a second user, such that the second user may experience the first portion from the location of the second user, such that aspects of the first user virtual world are effectively passed to the second user.

System and method for augmented and virtual reality

Method of rendering using a display device

Growing community of inventors

Hollywood, FL, United States of America

Samuel A Miller