Disney Enterprises, Inc.

Meta Platforms Technologies, LLC

Facebook Technologies, LLC

Eidgenossische Technische Hochschule Zurich

Eth Zurich

Samsung Electronics Co., Ltd.

Meta Platforms, Inc.

Liberovision Ag

Vizrt Ag

Alexander Sorkine Hornung

Gioacchino Noris

Oliver Wang

Markus H Gross

Gaurav Chaurasia

Henning Zimmer

Alexandru-Eugen Ichim

Christopher Richard Schroers

Sarthak Ray

Federico Perazzi

Alessia Marra

Panya Inversin

Manuel Lang

Yael Pritch

Jan Oberländer

Markus Gross

Robert Walker Sumner

Jennifer Lynn Spurlock

Jeffrey Witthuhn

Manuel Werlberger

William Arthur Hugh Steptoe

Derrick Readinger

Björn Wanbo

Fangwei Lee

Sheng Shen

Matthew James Alderman

Jonathan Mallinson

Federico Ruiz

Larissa Heike Laich

Javier Alejandro Sierra Santos

Manuel Ricardo Freire Santos

Aleksejs Anpilogovs

Connor Treacy

Marcus Tanner

Simon Heinzle

James Allan Booth

Jeffrey Wai-King Lock

Jonathan Richard Mayoh

Wojciech Matusik

Michael James LeBeau

Fabrice Pierre Armand Rousselle

Thomas Oskam

Maryann Simmons

Changil Kim

Fabian Langguth

Simone Meyer

Philipp Kraehenbuehl

Olga Sorkine Hornung

Do Kyoon Kim

Stelian Coros

Aljoscha Smolic

Nikolce Stefanoski

Aljosa A Smolic

Brian McWilliams

Hyun Jung Shim

Brian Whited

Jae Young Sim

Jeong Hwan Ahn

David M Adler

Marc Pollefeys

Arthur Nieuwoudt

Jean-Charles Bazin

Gian Diego Tipaldi

Wojciech Krzysztof Jarosz

Rolf Adelsberger

Wenzel Jakob

Alex Locher

Benjamin Resch

Hendrik Lensch

Lanny Starkes Smoot

Quinn Smithwick

Thomas R Gross

Abdelaziz Djelouah

Gregory Mayo Daly

Paul A Beardsley

Jan Novak

Aljosa Aleksej Andrej Smolic

Mahdi Salmani Rahimi

Andreas Breitenmoser

Anselm Grundhofer

Kenny Mitchell

Ariel Shamir

Max Grosse

Mario Botsch

Sebastian Sztuk

Nicola V Ranieri

Fabio Zünd

Kaan Yücer

Huw David Bowles

Remo Ziegler

Javier Alonso Mora

Lingwen Gan

Jeng-Weei Lin

Christoph Niederberger

Paul Timothy Furgale

Marcel Germann

Richard Keiser

Michael James Lebeau

Yifan Wang

A method includes receiving scene image data, comprising left and right images for each eye. Through an eye tracking module, the method includes determining the user's gaze direction or eye vergence. Using this information, the method includes identifying an object in the scene that the user is focusing on, and determining, using a depth estimation module, a left depth from the left eye to the object and a right depth from the right eye to the object. Further, based on the computed left and right depths, the method includes generating, for the left and right eye, constant left and right depth meshes, generating a left and right output images by projecting the left and right images on the corresponding constant left and right depth meshes. Additionally, the method includes displaying the left output image for the left eye and displaying the right output image for the right eye.

Eye-tracking assisted passthrough rendering

In particular embodiments, a computing system may divide at least a portion of a physical space surrounding a user into a plurality of three-dimensional (3D) regions, wherein each of the 3D regions is associated with an area of a plurality of areas in a plane. The system may generate estimated locations of features of objects in the portion of the physical space. Based on the estimated locations, the system may determine an occupancy state of each of the plurality of 3D regions. Then based on the occupancy states of the plurality of 3D regions, the system may determine that one or more of the plurality of areas have respective airspaces that are likely unoccupied by objects.

Systems, methods, and media for detecting object-free space

In one embodiment, a computing system may monitor one or more regions of a first 3D model of an environment to determine a frequency the respective region changes. The computing system may determine a first region of the one or more regions is static based on the frequency the respective region changes. The computing system may delay a first time period to a second time period to update the first 3D model based on comparisons between a second 3D model and first depth measurements of the first region. The computing system may detect whether the first region changed after the second time period based on comparisons between the second 3D model and the first depth measurements of the first region. The computing system may in response to detecting a change in the first region, update the first 3D model of the first region.

Adaptive model updates for dynamic and static scenes

A system generates a plurality of spatial points based on depth measurements of physical objects. The system determines, based on the plurality of spatial points, an occupancy score for each voxel within a plurality of voxels. The system identifies, based on a gaze of the user, a first set of occupied voxels that are in a field of view of the user and a second set of occupied voxels that are outside the field of view of the user. The system updates the occupancy scores of the first set of occupied voxels by temporally decaying one or more of the plurality of spatial points within the first set of occupied voxels. The system maintains the occupancy scores of the second set of occupied voxels. The system detects intrusions in a predefined subspace within a physical space based on the updated occupancy scores of the first set of occupied voxels.

Systems and methods for detecting objects within the boundary of a defined space while in artificial reality

Aspects of the present disclosure are directed to creating and administering artificial reality collaborative working environments and providing interaction modes for them. An XR work system can provide and control such artificial reality collaborative working environments to enable, for example, A) links between real-world surfaces and XR surfaces; B) links between multiple real-world areas to XR areas with dedicated functionality; C) maintaining access, while inside the artificial reality working environment, to real-world work tools such as the user's computer screen and keyboard; D) various hand and controller modes for different interaction and collaboration modalities; E) use-based, multi-desk collaborative room configurations; and F) context-based auto population of users and content items into the artificial reality working environment.

Artificial reality collaborative working environments

A method includes receiving video data of a user, the video data comprising a first captured image and a second captured image, generating a two-dimensional planar proxy of the user, determining a pose comprising a location and orientation of the two-dimensional planar proxy within a three-dimensional virtual environment, rendering one or more display images for one or more displays of an artificial-reality device based on the two-dimensional planar proxy having the determined pose and at least one of the first and second captured images, displaying the rendered one or more display images using the one or more displays, respectively, determining that a viewing angle of the artificial-reality device relative to the two-dimensional planar proxy exceeds a predetermined maximum threshold, and based on the determination that the viewing angle exceeds the predetermined maximum threshold, ceasing to display the one or more display images.

Systems and methods for displaying stereoscopic rendered image data captured from multiple perspectives

A mixed reality (MR) simulation system includes a console and a head mounted device (HMD). The MR system captures stereoscopic images from a real-world environment using outward-facing stereoscopic cameras mounted to the HMD. The MR system preprocesses the stereoscopic images to maximize contrast and then extracts a set of features from those images, including edges or corners, among others. For each feature, the MR system generates one or more two-dimensional (2D) polylines. Then, the MR system triangulates between 2D polylines found in right side images and corresponding 2D polylines found in left side images to generate a set of 3D polylines. The MR system interpolates between 3D vertices included in the 3D polylines or extrapolates additional 3D vertices, thereby generating a geometric reconstruction of the real-world environment. The MR system may map textures derived from the real-world environment onto the geometric representation faster than the geometric reconstruction is updated.

Reconstruction of essential visual cues in mixed reality applications

In one embodiment, a computing system may update a first 3D model of a region of an environment based on comparisons between the first 3D model and first depth measurements of the region generated during a first time period. The computing system may determine that the region is static by comparing the first 3D model to second depth measurements of the region generated during a second time period. The computing system may in response to determining that the region is static, detect whether the region changed after the second time period based on comparisons between a second 3D model of the region and third depth measurements of the region generated after the second time period, the second 3D model having a lower resolution than the first 3D model. The computing system may in response to detecting a change in the region, update the first 3D model of the region.

A method includes receiving, through a network by a computing system associated with an artificial-reality device, video data of a user of a second computing system comprising a first and second image of the user. The first and second images may be captured concurrently by a first camera and a second camera of the second computing system, respectively. The computing system generates a planar proxy for displaying the user and determines a pose for the planar proxy within a three-dimensional virtual environment. The computing system renders a left image for a left-eye display and a right image for a right-eye display of the artificial-reality device based on the planar proxy having the determined pose and the first image, and the planar proxy having the determined pose and the second image, respectively. The computing system displays the rendered left image and right image using the left-eye display and right-eye display, respectively.

Growing community of inventors

Zurich, Switzerland

Alexander Sorkine Hornung