Facebook Technologies, LLC

Meta Platforms Technologies, LLC

Meta Platforms, Inc.

Facebook, Inc.

Mark Terrano

Charlene Mary Atlas

Caryn Vainio

Chad Austin Bramwell

Kevin James MacKenzie

Ian Erkelens

Xiaoyu Liu

Systems, methods, and non-transitory computer-readable media are disclosed for selectively rendering augmented reality content based on predictions regarding a user's ability to visually process the augmented reality content. For instance, the disclosed systems can identify eye tracking information for a user at an initial time. Moreover, the disclosed systems can predict a change in an ability of the user to visually process an augmented reality element at a future time based on the eye tracking information. Additionally, the disclosed systems can selectively render the augmented reality element at the future time based on the predicted change in the ability of the user to visually process the augmented reality element.

Reducing head mounted display power consumption and heat generation through predictive rendering of content

The present disclosure relates to systems, methods, and non-transitory computer-readable media that initiate communication between users of a networking system within an extended reality environment. For example, the disclosed systems can generate an extended-reality lobby window graphical user interface element for display on an extended-reality device of a user. The disclosed systems can further determine a connection between the user and a co-user and provide an animated visual representation of the co-user for display within the extended-reality lobby window graphical user interface element. In response to receiving user input targeting the animated visual representation of the co-user, the disclosed systems can generate and send, for display on an extended-reality device of the co-user, an invitation to join an extended-reality communication session with the user.

Generating an extended-reality lobby window for communication between networking system users

The present disclosure relates to methods, systems, and non-transitory computer-readable media for using inter-device communications and user permissions to capture and transmit digital images of users who have opted in for other computing devices capturing their images at an event. For example, the disclosed systems can receive registration information corresponding to a user attending an event, including whether the user opts in to having digital images of the user captured during the event. During the event, the disclosed systems can provide the current location of the user to other computing devices at the event. Upon detecting a trigger, one or more of the computing devices can capture (and encrypt) a digital image of the user and provide the digital image for access by the user. The disclosed systems can also modify a digital image of the user to censor an image of the user from the digital image or prevent computing devices from capturing digital images of the user entirely based on indications provided by the user.

Utilizing inter-device communications and user permissions for capturing digital images at events

Systems, methods, and non-transitory computer-readable media are disclosed for variably rendering graphical representations of co-users in VR environments based on social graph information between the co-users and a VR user. For example, the disclosed systems can identify a co-user within a VR environment. Furthermore, the disclosed systems can determine a social relevancy (e.g., a social relationship type) between the co-user and a VR user within the VR environment based on social graph information. Then, the disclosed systems can select and/or determine a graphical representation and/or other capabilities of the co-user within the VR environment based on the social relevancy. Additionally, the disclosed systems can display the co-user within the VR environment using the determined graphical representation type (e.g., from the perspective of the VR user).

Using social connections to define graphical representations of users in an artificial reality setting

This disclosure describes an artificial reality system that presents artificial reality content in the context of a physical environment that includes a mirror or other reflective surface. In one example, this disclosure describes a method that includes capturing capture data representative of a physical environment, wherein the physical environment includes a reflective surface and a plurality of objects, determining a pose of the HMD, determining a map of the physical environment, wherein the map includes position information about the reflective surface and position information about each of the plurality of physical objects in the physical environment, identifying a visible object from among the plurality of physical objects, and generating artificial reality content associated with the visible object.

Movement instruction using a mirror in an artificial reality environment

Performing operations using a mirror in an artificial reality environment

Presenting artificial reality content using a mirror

In some examples, a virtual surface presented by an artificial reality (AR) system may be a virtual drawing surface in an artificial reality environment, with which users may interact using a physical peripheral device. For example, the AR system may enable a user to draw or write on a surface of the peripheral device and may simultaneously, i.e., along with the user inputs, render virtual markings corresponding to the user inputs on a virtual drawing surface locked to the surface of the peripheral device. In some cases, the AR system enables the user to 'transfer' the virtual markings rendered on the surface of the peripheral device (e.g., a virtual drawing) to another virtual drawing surface (e.g., to a planar or other surface in the artificial reality environment).

Virtual drawing surface interaction using a peripheral device in artificial reality environments

In general, the disclosure describes artificial reality (AR) systems and techniques for generating and presenting virtual surfaces within an artificial reality environment and for facilitating user interaction with the virtual surfaces using a physical peripheral device. For example, AR systems are described that generate and render virtual surfaces, such as a virtual pinboard or a virtual drawing surface (e.g., a virtual canvas) in an artificial reality environment, for display to a user. The AR systems enable the user to interact with the virtual surfaces using a physical peripheral device, which may be manipulated and otherwise interacted with by the user to provide input to an AR system through pose tracking of the peripheral device and/or via one or more input devices of the peripheral device, such as a presence-sensitive surface.

Virtual pinboard interaction using a peripheral device in artificial reality environments

An artificial reality system includes a head mounted display (HMD) and a compass generator that generates a proximity compass used to locate virtual or physical elements within an artificial reality environment. The artificial reality system can render a proximity compass around or in proximity to a virtual device, a virtual or physical object, or at a designated location. The proximity compass includes graphical elements that can represent virtual experiences (applications, games, utilities etc.), physical or virtual objects, and physical or virtual locations within the artificial reality environment. The positioning of a graphical element within the proximity compass represents a direction of the associated experience, object or location with respect to the HMD. The graphical elements within the proximity compass may rotate around a virtual hand-held device when the user changes the orientation of the hand-held device. The graphical elements may include data that identifies the experience associated with the graphical element.

Virtual proximity compass for navigating artificial reality environments

In some examples, a method comprises: obtaining, by an artificial reality system including a head mounted display (HMD), image data via an image capture device, the HMD configured to output artificial reality content; detecting, by the artificial reality system, a physical peripheral device from the image data; detecting, by the artificial reality system, a pose of the peripheral device; generating, by the artificial reality system, based on the pose of the peripheral device, a virtual pointer along at least a portion of a line between the peripheral device and a virtual surface, the virtual pointer pointing at a location of the virtual surface; performing, by the artificial reality system, one or more actions based on the location of the virtual surface; and rendering, by the artificial reality system, the virtual pointer and the virtual surface for display at the HMD.

Pointer-based interaction with a virtual surface using a peripheral device in artificial reality environments

The present disclosure relates to systems, non-transitory computer-readable media, and methods for improving hand input detection in a virtual environment based on small and/or rapid hand inputs. For instance, a hybrid hand input detection system can utilize a hybrid hand input model that incorporates a machine-learning hand pose model with an optical flow network to recognize fine degrees of hand movement corresponding to input commands within a virtual environment. In this manner, the hybrid hand input detection system can utilize a hybrid hand input model to recognize small and rapid hand inputs in virtual reality or augmented reality without requiring a physical input device.

Growing community of inventors

Redmond, WA, United States of America

Mark Terrano