Facebook Technologies, LLC

Oculus, Vr, LLC

Facebook, Inc.

Oskar Linde

Nirav Rajendra Patel

Sam Padinjaremannil Alex

Qi Qu

Ryan Hamilton Brown

Lyle David Bainbridge

Ali Yazdan

Hongyu Zhou

Matthew James Devoe

Ganesh Venkatraman

Andrew Melim

Dov Katz

Samuel Redmond D'Amico

Rasmus Karl Emil Simander

Kjell Erik Alexander Sjoholm

Joseph Patrick Sullivan

Robin Michael Miller

Simon Hallam

Julian James Hammerstein

Chengyuan Yan

Dong Yang

Kieran Tobias Levin

Adam Hewko

Neil Konzen

Lars Anders Bond

Christian Forster

Maksym Katsev

Eldad Yitzhak

Jijie Xu

James Aldrich

Kenneth Chi Kin Ng

Steven G McClellan

Joern Rehder

Janosch Nikolic

Andrew Melin

Embodiments of the present disclosure support a head-mounted display (HMD) wirelessly coupled to a console. The HMD includes a positional tracking system, a beam controller and a transceiver. The positional tracking system tracks position of the HMD and generates positional information describing the tracked position of the HMD. The transceiver communicates with a console via a wireless channel, in accordance with communication instructions, the communication instructions causing the transceiver to communicate over one directional beam of a plurality of directional beams. The beam controller determines a change in the positional information. Based on the change to the positional information, the beam controller determines a directional beam of the plurality of directional beams. The beam controller further generates the communication instructions identifying the determined directional beam, and provides the communication instructions to the transceiver.

Positional tracking assisted beam forming in wireless virtual reality systems

A calibration system includes a grid assembly, a platform, and a controller. The grid assembly includes at least one planar grid. The platform couples to a device under test (DUT) and move the DUT to a plurality of test positions in accordance with a motion sequence. Each test position is reached by a rotation about at most two different axes, and as the DUT moves through the motion sequence, at least one camera on the DUT captures image information describing portions of the grid assembly and an inertial measurement unit (IMU) on the DUT captures IMU information. The controller determines calibration information for the at least one camera on the DUT and for the IMU based in part on a parameterized model of the motion sequence of the DUT, the captured image information, and the captured IMU information.

Calibration system for concurrent calibration of device sensors

The disclosed computer-implemented method for offloading image-based tracking operations from a general processing unit to a hardware accelerator unit may include (1) sending imaging data from an imaging device to a hardware accelerator unit, and (2) directing the hardware accelerator unit to generate a multi-scale representation of the imaging data sent from the imaging device, (3) preparing a set of input data for a set of image-based tracking operations, and (4) directing the hardware accelerator unit to execute the set of image-based tracking operations using the generated multi-scale representation of the imaging data and the prepared set of input data. Various other methods, systems, and computer-readable media are also disclosed.

Systems and methods for offloading image-based tracking operations from a general processing unit to a hardware accelerator unit

An artificial reality system is described includes a hand-held controller tracking sub-system having two components, a Field-of-View (FOV) tracker and a non-FOV tracker that applies specialized motion models when one or more of controllers are not trackable within the field of view. In particular, under typical operating conditions, the FOV tracker receives state data for a Head Mounted Display (HMD) and controller state data (velocity, acceleration etc.) of a controller to compute estimated poses for the controller. If the controller is trackable (e.g., within the field of view and not occluded), then the pose as computed by the FOV tracker is used and the non-FOV tracker is bypassed. If the controller is not trackable within the field of view and the controller state data meets activation conditions for one or more corner tracking cases, then the non-FOV tracker applies one or more of specialized motion models to compute a controller pose for the controller.

Multi-layered artificial reality controller pose tracking architecture having prioritized motion models

A head-mounted display (HMD) is configured to capture images and/or video of a local area. The HMD includes an imaging assembly and a controller. The imaging assembly includes a plurality of cameras positioned at different locations on the HMD and oriented to capture images of different portions of a local area surrounding the HMD. The controller generates imaging instructions for each camera using image information. The imaging instructions cause respective midpoints of exposure times for each camera to occur at a same time value for each of the captured images. The cameras capture images of the local area in accordance with the imaging instructions. The controller determines a location of the HMD in the local area using the captured images and updates a model that represents a mapping function of the depth and exposure settings of the local area.

Head-mounted display tracking system

A virtual reality (VR) system tracks the location and position of a controller using image sensors on a headset and a controller. The headset and controller provide a first and second view of a user's environment. Using its camera, the headset generates a map of the environment and identifies its location within it. Based headset's location, the VR system generates a simulated world of the environment and displays it to the user. The headset also estimates the location of the controller. Based on the estimated location, the headset sends a portion of the map to the controller. The controller determines its pose using the portion of the map, the image sensors, and additional sensors. The controller sends its pose and an updated portion of the map to the headset. Based on the controller's pose and updated portion of the map, the VR system modifies the content displayed to the user.

Tracking using controller cameras

A calibration system is configured to determine calibration information of a head-mounted display (HMD). The calibration system comprises a first, second, and third planar grid, a movable platform, and a calibration controller. Each planar grid includes a plurality of fiducial markers that are displayed in accordance with a display pattern. The HMD is coupled to the movable platform, which moves the HMD before the planar grids as a plurality of cameras on the HMD captures images of the planar grids with fiducial markers. The calibration controller controls a motion sequence of the movable platform and determines calibration information for each of the cameras on the HMD and calibration information for an inertial measurement unit (IMU) within the HMD. The calibration information is based in part on a parameterized model of the motion sequence of the HMD.

Calibration system for a head-mounted display tracking system

A virtual reality (VR) system tracks the position of a controller. The VR system includes an image tracking system comprising of a number of fixed cameras, and a headset worn by the user that includes an imaging device to capture images of a controller operated by the user. The controller includes a set of features disposed on the surface of the controller. The image tracking system provides a first view of the controller. The imaging device mounted on the headset provides a second view of the controller. Each view of the controller (i.e., from the headset and from the image tracking system) provides a distinct set of features observed on the controller. The first and second sets of features are identified from the captured images and a pose of the controller is determined using the first set of features and the second set of features.

Tracking controllers of a virtual reality system

Head-mounted display

A virtual reality (VR) headset includes a first camera and a second camera capturing image data of an environment of the VR headset. Each camera has a field of view, and a portion of the fields of view of the first and second cameras overlap while a portion of the fields of view do not overlap. A processor receiving the image data from the first and second cameras is configured to identify a first observation of a position of the VR headset in the environment and positions of a plurality of features based on the image data captured by the first camera. The processor also identifies a second observation of the position of the VR headset in the environment and the positions of the features based on the image data captured by the second camera. Based on the first and second observations, the processor determines a model of the environment.

Non-overlapped stereo imaging for virtual reality headset tracking

Growing community of inventors

Belmont, CA, United States of America

Oskar Linde