Facebook Technologies, LLC

Meta Platforms Technologies, LLC

Ioannis Pavlidis

Jeffrey Witthuhn

Meghana Reddy Guduru

Ananth Ranganathan

Gregory L Smith

Shuang Li

Oytun Akman

Xiang Wei

Sarah Tanner Simpson

Bodhi Keanu Donselaar

Kevin Joseph Sheridan

Kevin Xiao

Federico Adrian Camposeco Paulsen

Peter Koch

Wenliang Zhao

Ying-Chieh Huang

Andrew Melim

In one embodiment, a method includes capturing images of a first user wearing a VR display device in a real-world environment. The method includes receiving a VR rendering of a VR environment. The VR rendering is from the perspective of the mobile computing device with respect to the VR display device. The method includes generating a first MR rendering of the first user in the VR environment. The first MR rendering of the first user is based on a compositing of the images of the first user and the VR rendering. The method includes receiving an indication of a user interaction with one or more elements of the VR environment in the first MR rendering. The method includes generating, in real-time responsive to the indication of the user interaction with the one or more elements, a second MR rendering of the first user in the VR environment. The one or more elements are modified according to the interaction.

Dynamic mixed reality content in virtual reality

In one embodiment, a method includes using one or more cameras of a mobile computing device to capture one or more images of a first user wearing a VR display device in a real-world environment. The mobile computing device transmits a pose of the mobile computing device with respect to the VR display device to a VR system. The mobile computing device receives from the VR system a VR rendering of a VR environment. The VR rendering is from the perspective of the mobile computing device with respect to the VR display device. The method includes segmenting the first user from the one or more images and generating, in real-time responsive to capturing the one or more images, a MR rendering of the first user in the VR environment. The MR rendering of the first user is based on a compositing of the segmented one or more images of the first user and the VR rendering.

In general, this disclosure describes an artificial reality system that provides asymmetric user experiences to users associated with user devices that operate according to different modes of engagement with the artificial reality system. Different user devices may have different capabilities, be used by users having different roles for an artificial reality application, or otherwise be configured to interact in a variety of ways with an artificial reality system.

Artificial reality system having multiple modes of engagement

An artificial reality system is described that implements adaptive degrees-of-freedom (DOF) selection when tracking frames of reference and rendering artificial reality content. In one example, the artificial reality system comprises a head mounted display (HMD) that outputs rendered artificial reality content. A performance monitor determines one or more performance indicators associated with the artificial reality system. A degree-of-freedom (DOF) selector applies one or more policies to the performance indicators to select between a first mode in which a pose tracker computes one or more poses of the HMD within the 3D environment using 6DOF and a second mode in which the pose tracker computes the one or more poses using 3DOF. The pose tracker computes the one or more poses for the HMD within the 3D environment in accordance with the selected mode. A rendering engine renders the content for the artificial reality application based on the computed pose.

Artificial reality system having adaptive degrees of freedom (DOF) selection

Growing community of inventors

Newark, CA, United States of America

Ioannis Pavlidis