Microsoft Technology Licensing, LLC

Facebook, Inc.

Meta Platforms Technologies, LLC

Facebook Technologies, LLC

Alvaro Collet Romea

Ming Chuang

Yuheng Ren

Jakob Julian Engel

Xiaoyang Gao

Christopher Sweeney

Jiawen Zhang

Jingming Dong

Steve Sullivan

Adam G Kirk

Krishnan Ramnath

Raul Mur Artal

Pat Sweeney

Hugues Herve Hoppe

Zheng Wang

Kristofer Noel Iverson

Emmett Walter Lalish

Hermes Germi Pique Corchs

Don Gillett

William Crow

Patrick John Sweeney

Matthew Adam Simari

Steven Kish

Tullie Murrell

Thomas Ward Meyer

Fabian Andres Prada Nino

Steven Craig Sullivan

Bao Zhang

Dennis Evseev

Jiao Li

The disclosed computer-implemented method may include acquiring, from a client device within a real-world environment, information representative of the real-world environment, and transmitting the information representative of the real-world environment to a relocalization service. The method may further include receiving, from the relocalization service, (1) an anchor point that may include a mapped position within the real-world environment, and (2) a determined position within the real-world environment of a client device relative to the mapped position of the anchor point. The method may further include sending an identifier of the anchor point to an asset management service, and obtaining, from the asset management service, a digital asset. The method may further include presenting the digital asset at a position within an artificial environment relative to the mapped position of the anchor point. Various other methods, systems, and computer-readable media are also disclosed.

Systems and methods for presenting digital assets within artificial environments via a loosely coupled relocalization service and asset management service

The subject disclosure is directed towards a framework that is configured to allow different background-foreground segmentation modalities to contribute towards segmentation. In one aspect, pixels are processed based upon RGB background separation, chroma keying, IR background separation, current depth versus background depth and current depth versus threshold background depth modalities. Each modality may contribute as a factor that the framework combines to determine a probability as to whether a pixel is foreground or background. The probabilities are fed into a global segmentation framework to obtain a segmented image.

Multimodal foreground background segmentation

In one embodiment, a method includes generating, by a device, first tracking data using a first tracking algorithm, based on first video frames associated with a scene. An augmented-reality (AR) effect may be displayed based on the first tracking data. The device may generate a first confidence score associated with the first tracking data and determine that the first confidence score is above a threshold. The device may generate, based on second video frames subsequent to the first video frames, second tracking data using the first tracking algorithm. The device may determine that an associated second confidence score is below a threshold. In response, the device may generate, based on third video frames subsequent to the second video frames, third tracking data using a second tracking algorithm different from the first. The device may then display the AR effect based on the third tracking data.

Dynamic graceful degradation of augmented-reality effects

In one embodiment, a method includes determining, using one or more location sensors of a computing device, an approximate location of the computing device, identifying a content object located within a threshold distance of the approximate location, wherein an augmented-reality map associates the content object with a stored model of a real-world object and specifies a location of the content object on or relative to the stored model of the real-world object, obtaining an image from a camera of the device, identifying, in the image, a target real-world object that matches the stored model of the real-world object, determining a content object location based on a location of the target real-world object and the location of the content object on or relative to the model of the real-world object, and displaying the content object at the content object location.

Mobile persistent augmented-reality experiences

Systems, devices, and methods are described herein for transforming three dimensional (3D) video data into a 3D printable model. In one aspect, a method for transforming 3D video data may include receiving 3D video data indicated or selected for 3D printing. The selected portion or 3D video data, which may include a frame of the 3D video data, may be repaired or modified to generate a 3D model that define at least one enclosed volume. At least one of the enclosed volumes of the 3D video data may be re-oriented based on at least one capability of a target 3D printing device. In some aspects, the re-orienting may be performed to optimize at least one of a total print volume or print orientation of the at least one enclosed volume. In some aspects, the method may be performed in response to a single selection or action performed by a user.

3D printing using 3D video data

In some examples, a computing device can determine synthetic meshes based on source meshes of a source mesh sequence and target meshes of a target mesh sequence. The computing device can then place the respective synthetic meshes based at least in part on a rigid transformation to define a processor-generated character. For example, the computing device can determine subsets of the mesh sequences based on a similarity criterion. The computing device can determine modified first and second meshes having a connectivity corresponding to a reference mesh. The computing device can then determine the synthetic meshes based on the modified first and second meshes. In some examples, the computing device can project source and target textures onto the synthetic mesh to provide projected source and target textures. The computing device can determine a synthetic texture registered to the synthetic mesh based on the projected source and target textures.

Motion capture and character synthesis

A computing system is configured for context-adaptive allocation of render model resources that may sacrifice some level of detail in a computational description of a 3D scene before rendering in order to accommodate resource limitations in a rendering environment such as available processor cycles, and/or bandwidth for data transmission to a processor. Such resource limitations can often preclude rendering a richly detailed 3D scene, particularly in full-motion and/or in real time. An importance function describing the relative perceptual importance of elements that make up the 3D scene is utilized to enable resources to be adaptively allocated so that more resources go to visual elements of the 3D scene that have a higher perceptual importance. The rendered output may thus optimize visual fidelity for the computational description within the resource constrained rendering environment.

Context-adaptive allocation of render model resources

Consistent tessellation via topology-aware surface tracking is provided in which a series of meshes is approximated by taking one or more meshes from the series and calculating a transformation field to transform the keyframe mesh into each mesh of the series, and substituting the transformed keyframe meshes for the original meshes. The keyframe mesh may be selected advisedly based upon a scoring metric. An error measurement on the transformed keyframe exceeding tolerance or threshold may suggest another keyframe be selected for one or more frames in the series. The sequence of frames may be divided into a number of subsequences to permit parallel processing, including two or more recursive levels of keyframe substitution. The transformed keyframe meshes achieve more consistent tessellation of the object across the series.

Consistent tessellation via topology-aware surface tracking

A three-dimensional (3D) scene is computationally reconstructed using a combination of plural modeling techniques. Point clouds representing an object in the 3D scene are generated by different modeled techniques and each point is encoded with a confidence value which reflects a degree of accuracy in describing the surface of the object in the 3D scene based on strengths and weaknesses of each modeling technique. The point clouds are merged in which a point for each location on the object is selected according to the modeling technique that provides the highest confidence.

Mixed three dimensional scene reconstruction from plural surface models

Growing community of inventors

Seattle, WA, United States of America

Alvaro Collet Romea