Facebook Technologies, LLC

Meta Platforms Technologies, LLC

Chad Austin Bramwell

Mark Terrano

Charlene Mary Atlas

Caryn Vainio

Xiaoyu Liu

This disclosure describes an artificial reality system that presents artificial reality content in the context of a physical environment that includes a mirror or other reflective surface. In one example, this disclosure describes a method that includes capturing capture data representative of a physical environment, wherein the physical environment includes a reflective surface and a plurality of objects, determining a pose of the HMD, determining a map of the physical environment, wherein the map includes position information about the reflective surface and position information about each of the plurality of physical objects in the physical environment, identifying a visible object from among the plurality of physical objects, and generating artificial reality content associated with the visible object.

Movement instruction using a mirror in an artificial reality environment

Performing operations using a mirror in an artificial reality environment

Presenting artificial reality content using a mirror

In some examples, a virtual surface presented by an artificial reality (AR) system may be a virtual drawing surface in an artificial reality environment, with which users may interact using a physical peripheral device. For example, the AR system may enable a user to draw or write on a surface of the peripheral device and may simultaneously, i.e., along with the user inputs, render virtual markings corresponding to the user inputs on a virtual drawing surface locked to the surface of the peripheral device. In some cases, the AR system enables the user to 'transfer' the virtual markings rendered on the surface of the peripheral device (e.g., a virtual drawing) to another virtual drawing surface (e.g., to a planar or other surface in the artificial reality environment).

Virtual drawing surface interaction using a peripheral device in artificial reality environments

In general, the disclosure describes artificial reality (AR) systems and techniques for generating and presenting virtual surfaces within an artificial reality environment and for facilitating user interaction with the virtual surfaces using a physical peripheral device. For example, AR systems are described that generate and render virtual surfaces, such as a virtual pinboard or a virtual drawing surface (e.g., a virtual canvas) in an artificial reality environment, for display to a user. The AR systems enable the user to interact with the virtual surfaces using a physical peripheral device, which may be manipulated and otherwise interacted with by the user to provide input to an AR system through pose tracking of the peripheral device and/or via one or more input devices of the peripheral device, such as a presence-sensitive surface.

Virtual pinboard interaction using a peripheral device in artificial reality environments

An artificial reality system includes a head mounted display (HMD) and a compass generator that generates a proximity compass used to locate virtual or physical elements within an artificial reality environment. The artificial reality system can render a proximity compass around or in proximity to a virtual device, a virtual or physical object, or at a designated location. The proximity compass includes graphical elements that can represent virtual experiences (applications, games, utilities etc.), physical or virtual objects, and physical or virtual locations within the artificial reality environment. The positioning of a graphical element within the proximity compass represents a direction of the associated experience, object or location with respect to the HMD. The graphical elements within the proximity compass may rotate around a virtual hand-held device when the user changes the orientation of the hand-held device. The graphical elements may include data that identifies the experience associated with the graphical element.

Virtual proximity compass for navigating artificial reality environments

In some examples, a method comprises: obtaining, by an artificial reality system including a head mounted display (HMD), image data via an image capture device, the HMD configured to output artificial reality content; detecting, by the artificial reality system, a physical peripheral device from the image data; detecting, by the artificial reality system, a pose of the peripheral device; generating, by the artificial reality system, based on the pose of the peripheral device, a virtual pointer along at least a portion of a line between the peripheral device and a virtual surface, the virtual pointer pointing at a location of the virtual surface; performing, by the artificial reality system, one or more actions based on the location of the virtual surface; and rendering, by the artificial reality system, the virtual pointer and the virtual surface for display at the HMD.

Growing community of inventors

Sammamish, WA, United States of America

Chad Austin Bramwell