Oculus, Vr, LLC

Facebook Technologies, LLC

Dov Katz

Simon Hallam

Jennifer Dolson

Michael John Toksvig

Oskar Linde

Kieran Tobias Levin

Andrew Melim

Neil Konzen

Torin Ross Herndon

Maksym Katsev

Klas Petter Ivmark

Eric Loren Vaughan

Timothy Paul Omernick

Ziheng Wang

Robin Michael Miller

Rasmus Karl Emil Simander

Kjell Erik Alexander Sjoholm

Jonathan Shine

Michael Antonov

Evan Mark Richards

Shizhe Shen

Nadav Grossinger

Jason Andrew Higgins

Palmer Freeman Luckey

Samuel Redmond D'Amico

Steve LaValle

Paul Austin Buckley

Steven LaValle

Steve Lavalle

Sam Redmond D'AMICO

Jennifer Leigh Dolson

A depth camera assembly (DCA) configured to determine distances between the headset and one or more objects in an area surrounding the headset. The DCA includes an imaging device, an illumination source, and a controller. The controller identifies objects in a portion of the local area, determines a depth zone for each object and corresponding structured light (SL) illumination parameters including a SL pattern for each object based on the depth zone, instructs the illumination source to illuminate a scene comprising the one or more objects with the determined SL pattern, and instructs the imaging device to capture images of the illuminated objects. The controller determines the depth information for the illuminated objects and updates the depth information associated with the objects.

Multi-pattern depth camera assembly

A virtual reality system includes a head-mounted display (HMD) having one or more facial sensors and illumination sources mounted to a surface of the HMD. For example, the facial sensors are image capture devices coupled to a bottom side of the HMD. The illumination sources illuminate portions of a user's face outside of the HMD, while the facial sensors capture images of the illuminated portions of the user's face. A controller receives the captured images and generates a representation of the portions of the user's face by identifying landmarks of the user's face in the captured images and performing other suitable image processing methods. Based on the representation, the controller or another component of the virtual reality system generates content for presentation to the user.

Tracking portions of a user's face uncovered by a head mounted display worn by the user

A hand-held controller for a virtual-reality system includes an antenna, a grip, a cage coupled to the grip, and a plurality of LEDs on an outer surface of the cage. Wireless signals are received from a remote camera via the antenna. The plurality of LEDs is duty-cycled, which includes turning on the plurality of LEDs at times determined using the wireless signals. The times when the plurality of LEDs is turned on correspond to times when the remote camera captures images of the hand-held controller.

Hand-held controllers with light-emitting diodes synchronized to an external camera

A facial tracking system generates a virtual rendering of a portion of a face of a user wearing a head-mounted display (HMD). The facial tracking system illuminates portions of the face inside the HMD. The facial tracking system captures a plurality of facial data of the portion of the face using one or more facial sensors located inside the HMD. A plurality of planar sections of the portion of the face are identified based at least in part on the plurality of facial data. The plurality of planar sections are mapped to one or more landmarks of the face. Facial animation information is generated based at least in part on the mapping, the facial animation information describing a portion of a virtual face corresponding to the portion of the user's face.

Facial animation using facial sensors within a head-mounted display

A virtual reality (VR) headset calibration system calibrates a VR headset, which includes a plurality of locators and an inertial measurement unit (IMU) generating output signals indicative of motion of the VR headset. The system comprises a calibration controller configured to receive a headset model of the VR headset that identifies expected positions of each of the locators. The controller controls cameras to capture images of the VR headset while the headset is moved along a predetermined path. The images detect actual positions of the locators during the movement along the predetermined path. Calibration parameters for the locators are generated based on differences between the actual positions and the expected positions. Calibration parameters for the IMU are generated based on the calibration parameters for the locators and differences between expected and actual signals output by the IMU. The calibration parameters are stored to the VR headset.

Stereo-based calibration apparatus

A virtual reality (VR) system tracks the position of a controller. The VR system includes an image tracking system comprising of a number of fixed cameras, and a headset worn by the user that includes an imaging device to capture images of a controller operated by the user. The controller includes a set of features disposed on the surface of the controller. The image tracking system provides a first view of the controller. The imaging device mounted on the headset provides a second view of the controller. Each view of the controller (i.e., from the headset and from the image tracking system) provides a distinct set of features observed on the controller. The first and second sets of features are identified from the captured images and a pose of the controller is determined using the first set of features and the second set of features.

Tracking controllers of a virtual reality system

A virtual reality (VR) headset includes a first camera and a second camera capturing image data of an environment of the VR headset. Each camera has a field of view, and a portion of the fields of view of the first and second cameras overlap while a portion of the fields of view do not overlap. A processor receiving the image data from the first and second cameras is configured to identify a first observation of a position of the VR headset in the environment and positions of a plurality of features based on the image data captured by the first camera. The processor also identifies a second observation of the position of the VR headset in the environment and the positions of the features based on the image data captured by the second camera. Based on the first and second observations, the processor determines a model of the environment.

Non-overlapped stereo imaging for virtual reality headset tracking

A virtual reality (VR) includes a sparse projection system configured to generate a plurality of clusters using one or more diffractive optical elements. Each generated cluster has a unique configuration that corresponds to a unique location in a virtual mapping of a local area. The sparse projection system projects the generated clusters throughout the local area. A VR console receives a series of images of the local area from the imaging device, with at least one image includes at least one cluster. The VR console determines a location of a VR headset within the virtual mapping of the local area based at least in part on a configuration of the at least one cluster included in the series of images. Content is generated based at least in part on the determined location of the VR headset and is provided to the VR headset for presentation.

Sparse projection for a virtual reality system

A virtual reality (VR) console receives slow calibration data from an imaging device and fast calibration data from an inertial measurement unit on a VR headset including a front and a rear rigid body. The slow calibration data includes an image where only the locators on the rear rigid body are visible. An observed position is determined from the slow calibration data and a predicted position is determined from the fast calibration data. If a difference between the observed position and the predicted position is greater than a threshold value, the predicted position is adjusted by a temporary offset until the difference is less than the threshold value. The temporary offset is removed by re-calibrating the rear rigid body to the front rigid body once locators on both the front and rear rigid body are visible in an image in the slow calibration data.

Calibration of multiple rigid bodies in a virtual reality system

A head mounted display (HMD) in a VR system includes sensors for tracking the eyes and face of a user wearing the HMD. The VR system records calibration attributes such as landmarks of the face of the user. Light sources illuminate portions of the user's face covered by the HMD. In conjunction, facial sensors capture facial data. The VR system analyzes the facial data to determine the orientation of planar sections of the illuminated portions of face. The VR system aggregates planar sections of the face and maps the planar sections to landmarks of the face to generate a facial animation of the user, which can also include eye orientation information. The facial animation is represented as a virtual avatar and presented to the user.

Face and eye tracking using facial sensors within a head-mounted display

Embodiments relate to adjusting an image frame based on motion sensor data. An image frame including image data for a plurality of rows of the image frame is captured sequentially row-by-row. Motion sensor data corresponding to the image frame is also captured. Motion vectors describing motion between each row of the image frame are determined based on the motion sensor data. An image data translation is calculated for each row of image frame based on the motion vector for the row. The image frame is adjusted based on the image data translation for each row of the image frame to generate an adjusted image frame.

Rolling shutter blur reduction using motion data

A virtual reality (VR) console receives slow calibration data from an imaging device and fast calibration data from an inertial measurement unit on a virtual reality headset. Using a model of the VR headset, the VR console identifies model locators corresponding to locators on the VR headset and generates estimated positions for locators included in slow calibration data. The VR console adjusts calibration parameters so a relative distance between estimated positions of the locators and positions of their corresponding model locators is less than a threshold value. From the estimated positions, the VR console generates calibrated positions of a reference point on the VR headset associated with images from the slow calibration data. The VR console determines predicted positions of the reference point from the calibrated positions and adjusts calibration parameters so intermediate estimated positions of the reference point are within a threshold distance of the predicted positions.

Growing community of inventors

Irvine, CA, United States of America

Dov Katz