Apple Inc.

Other

Branko Petljanski

Noah Denis Bedard

Ricardo Jansson Motta

Kathrin Berkner Cieslicki

Qiong Huang

Brett D Miller

Daniel R Kurz

Raffi A Bedikian

Emanuele Mandelli

Li Jia

Kathrin Berkner-Cieslicki

Kenichi Saito

Daniel R Hennigan

Thomas Gebauer

John N Border

Jason C Sauers

Edward Allen Valko

Phil M Hobson

Andrew Kenneth John McMahon

Hyungryul J Choi

Graham Butler Myhre

Edward S Huo

Yoshikazu Shinohara

Yury Anatolievich Petrov

Paul V Johnson

Tom Sengelaub

Hua Gao

Victoria C Chan

Nan Zhu

William W Sprague

Martin Haller

Christina G Gambacorta

Minwoong Kim

Elijah H Kleeman

Brandon E Clarke

Keenan Molner

Andrei Nikiforov

Se Hoon Lim

Frame capture techniques may include obtaining, from at least one sensor, sensor data indicating a motion rate of the system in the environment, receiving a request for the image data of the environment, and selecting, by the controller, a frame rate based on one or more requesting client applications and the sensor data. In addition, the frame tracking techniques may include detecting, based on a motion parameter in the sensor data and by the controller, that a motion of the system falls below a predetermined threshold; and triggering frame capture by the camera in accordance with the sensor data and the selected frame rate, the frame capture being triggered in accordance with the request and the detection.

Systems and methods for triggering a frame capture in a wearable device

In one implementation, a method includes emitting light with modulating intensity from a plurality of light sources towards an eye of a user. The method includes receiving light intensity data indicative of an intensity of the plurality of glints reflected by the eye of the user in the form of a plurality of glints. The method includes determining an eye tracking characteristic of the user based on the light intensity data. In one implementation, a method includes generating, using an event camera comprising a plurality of light sensors at a plurality of respective locations, a plurality of event messages, each of the plurality of event messages being generated in response to a particular light sensor detecting a change in intensity of light and indicating a particular location of the particular light sensor. The method includes determining an eye tracking characteristic of a user based on the plurality of event messages.

Method and device for eye tracking using event camera data

In one implementation, an apparatus includes: a display to emit light in a first wavelength range; one or more light sources to emit light in a second wavelength range; a camera to detect the light in the second wavelength range; and an eyepiece to reflect and refract the light in the first wavelength range while passing, without substantial distortion, the light in the second wavelength range, wherein the eyepiece includes two lens halves separated by a retarder that changes light in the first wavelength range from a first polarization to a second polarization different from the first polarization.

Method and device for image display and eye tracking through a catadioptric lens

This disclosure describes a novel lens system having an entrance pupil at a location matching, or nearly matching, the eye's optical system entrance pupil. The described lens system is significantly thinner than a single-aperture lens with matched entrance pupil and is achieved by using an array of camera elements. Each camera element contains camera optics called 'lenslets' and a sensor (e.g., an image or depth sensor), where a lenslet can include one or more lens elements (e.g., a compound lens). Individual camera elements can be arranged such that THEIR optical axes intersect at, or near, the eye's entrance pupil location. In addition, each camera element's field-of-view (FOV) corresponds to a small sector of a large FOV. The FOVs of adjacent camera elements can be non- or slightly-overlapping so that a wide-angle image can be formed by concatenation of the individual camera element images.

Lens array for shifting perspective of an imaging system

A head-mounted display may include a display system and an optical system in a housing. The display system may have displays that produce images. Positioners may be used to move the displays relative to the eye positions of a user's eyes. An adjustable optical system may include tunable lenses such as tunable cylindrical liquid crystal lenses. The displays may be viewed through the lenses when the user's eyes are at the eye positions. A sensor may be incorporated into the head-mounted display to measure refractive errors in the user's eyes. The sensor may include waveguides and volume holograms, and a camera for gathering light that has reflected from the retinas of the user's eyes. Viewing comfort may be enhanced by adjusting display positions relative to the eye positions and/or by adjusting lens settings based on the content being presented on the display and/or measured refractive errors.

Head-mounted display device with vision correction

A device for MR/VR systems that includes a two-dimensional array of cameras that capture images of respective portions of a scene. The cameras are positioned along a spherical surface so that the cameras have adjacent fields of view. The entrance pupils of the cameras are positioned at or near the user's eye while the cameras also form optimized images at the sensor. Methods for reducing the number of cameras in an array, as well as methods for reducing the number of pixels read from the array and processed by the pipeline, are also described.

Camera system

Methods and apparatus for using polarized light (e.g., infrared (IR) light) to improve eye-related functions such as iris recognition. An eye camera system includes one or more IR cameras that capture images of the user's eyes that are processed to perform iris recognition, gaze tracking, or other functions. At least one polarizing element may be located in the path of the light which is used to capture images of the user's eye. The user's eye may be illuminated by IR light emitted by one or more LEDs. At least one of the LEDs may be an LED with a polarizing filter. Instead or in addition, at least one polarizer may be located at the eye camera sensor, in the eye camera optics, or as or in an additional optical element located on the light path between the eye camera and the user's eye.

Eye camera systems with polarized light

In one implementation, an apparatus includes a display to emit light in a first wavelength range, a camera to detect light in a second wavelength range, an eyepiece to distort light in the first wavelength range, and one or more light sources, disposed between the eyepiece and the display, to emit light in the second wavelength range. In one implementation, an apparatus includes a display to emit light in a first wavelength range, one or more light sources to emit light in a second wavelength range, a camera to detect light in a second wavelength range, and an eyepiece to reflect and refract light in the first wavelength range while passing, without substantial distortion, light in the second wavelength range.

A scene camera system that includes two or more mirrors that reflect light from a respective portion of a field of view (FOV) in front of the system and two or more cameras that each capture the light reflected by a respective one of the two or more mirrors. By using the mirrors to reflect the light, the cameras' entrance pupils are imaged to a location closer to a user's eyes to thus achieve a more accurate representation of the perspective of the user.

Mirror-based scene cameras

In one implementation, a method includes emitting light with modulating intensity from a plurality of light sources towards an eye of a user. The method includes receiving light intensity data indicative of an intensity of the plurality of glints reflected by the eye of the user in the form of a plurality of glints and determining an eye tracking characteristic of the user based on the light intensity data. In one implementation, a method includes generating, using an event camera comprising a plurality of light sensors at a plurality of respective locations, a plurality of event messages, each of the plurality of event messages being generated in response to a particular light sensor detecting a change in intensity of light and indicating a particular location of the particular light sensor. The method includes determining an eye tracking characteristic of a user based on the plurality of event messages.

An eye tracking system for detecting position and movements of a user's eyes in a head-mounted display (HMD). The eye tracking system may include at least one near-IR (NIR) eye tracking camera positioned at each side of the user's face and pointed towards eye-facing surfaces of eyepieces of the HMD, an illumination source that emits NIR light towards the user's eyes, and hot mirrors located between the eye-facing surfaces of the eyepieces and the user's eyes. The hot mirrors reflect at least a portion of NIR light, while allowing visible light to pass. The cameras capture images of the user's eyes reflected by the hot mirrors. Positioning the hot mirrors at or near the eye-facing surfaces of the eyepieces allows the cameras to be placed at the sides of the user's face without having to image through the eyepieces.

Eye tracking system

Growing community of inventors

San Jose, CA, United States of America

Branko Petljanski