Meta Platforms Technologies, LLC

Facebook Technologies, LLC

Jan Herling

Adrian Brian Ratter

Daniel Scharstein

Jan-Michael Frahm

Ahmad Al-Dahle

Nils Plath

In one embodiment, a method includes scanning a real-world environment with a first device associated with a first user; generating a three-dimensional model of the real-world environment, transmitting the three-dimensional model to a head-mounted device associated with the first user, determining a pose of the head-mounted device by localizing the head-mounted device within the three-dimensional model based on images captured by a second camera of the head-mounted device, displaying, on the head-mounted device, a virtual space corresponding to the scanned real-world environment generated based on the three-dimensional model as viewed from the pose, and transmitting, to a remote head-mounted device of a second user, data corresponding to the three-dimensional model and the pose of the head-mounted device, the data being configured for rendering, by the remote head-mounted device, the virtual space with a first avatar corresponding to the first user having the pose.

Method of generating a virtual environment by scanning a real-world environment with a first device and displaying the virtual environment on a second device

In one embodiment, a method includes the steps of capturing an image from a camera viewpoint, the image depicting a physical keyboard, detecting one or more shape features of the physical keyboard depicted in the image by comparing pixels of the image to a predetermined shape template, the predetermined shape template representing visual characteristics of spaces between keyboard keys, accessing predetermined shape features of a keyboard model associated with the physical keyboard, and determining a pose of the physical keyboard based on comparisons between (1) the detected one or more shape features of the physical keyboard and (2) projections of the predetermined shape features of the keyboard model toward the camera viewpoint.

Physical keyboard tracking

Disclosed herein are related to a system and a method for porting a physical object in a physical space into a virtual reality. In one approach, the method includes detecting an input device in a physical space relative to a user of the input device. In one approach, the method includes presenting, by a display device to the user, a virtual model of the detected input device in a virtual space at a location and an orientation. The location and the orientation of the virtual model in the virtual space may correspond to a location and an orientation of the input device in the physical space relative to the user. In one approach, the method includes visually providing relative to the virtual model in the virtual space, through the display device, spatial feedback on the user's interaction with the input device in the physical space.

Growing community of inventors

Seattle, WA, United States of America

Jan Herling