Facebook, Inc.

Facebook Technologies, LLC

Meta Platforms Technologies, LLC

Meta Platforms, Inc.

Yuanshuo Lu

Jonathan Lim

Mohmmad Feisal Saleh Amir Rasras

Alex Garcia Goncalves

Kyle Sejin Yoon

An augmented reality (AR) effect system can improve application of AR effects by sharing resources between AR effects. The AR effect system can employ manifests for AR effects that define which resources are required to render each AR effect. The AR effect system can organize rendering operations used by selected AR effects into a pipeline and can use the manifests of the AR effects to determine when each resource will be needed. Based on this pipeline, the AR effect system can create a cache order defining a resource schedule which specifies, when a resource is freed, conditions for whether to save the resource to a local cache or unload the resource. As rendering of the video with the AR effects progresses, the resource schedule can control whether resources not currently being used to render an AR effect should be unloaded or cached for fast access for future render operations.

Sharing of resources for generating augmented reality effects

In one embodiment, a method includes retrieving a video stream that was recorded while a first artificial-reality effect was being displayed on the video stream, where each frame of the video stream comprises a real-world scene without the first artificial-reality effect, retrieving an artificial-reality state information stream corresponding to the video stream, where the artificial-reality state information stream comprises state information associated with the first artificial-reality effect, retrieving one or more contextual data streams corresponding to the video stream, where the first artificial-reality effect displayed on the video stream was rendered based on at least a portion of the one or more contextual data streams, rendering a second artificial-reality effect based on at least a portion of the artificial-reality state information stream and a portion of the one or more contextual data streams, and displaying the second artificial-reality effect on the video stream.

Rendering post-capture artificial-reality effects based on artificial-reality state information

Augmented reality effect resource sharing

Growing community of inventors

Sunnyvale, CA, United States of America

Yuanshuo Lu