Snap Inc.

Harman International Industries, Inc.

Fuji Xerox Co., Ltd.

Sven Kratz

Stefan J Marti

Joseph Verbeke

Andrés Monroy-Hernández

Rajan Vaish

Yu Jiang Tham

Adam Boulanger

Brian Anthony Smith

Qiong Liu

Patrick Chiu

Fannie Liu

Lei Zhang

Shang Ma

Tianying Chen

Timothy Chong

Olivia Seow

Donald G Kimber

Ana Maria Cardenas Gasca

Neeka Mansourian

Joanne Leong

Xingyu Liu

Hanseul Jun

Anthony Dunnigan

Maribeth Joy Back

Priya Seshadri

Evgeny Burmistrov

James Vaughan

Ilteris Kaan Canberk

Davide Di Censo

Karen Stolzenberg

Laurent Denoue

David Meisenholder

Shin Hwun Kang

Jun Shingu

Daniel Avrahami

Ryota Mizutani

Yusuke Yamaura

Miteshkumar Patel

Tsung-Yu Tsai

Bee Yian Liew

Hao Tang

Joshua Sweers

Mingwen Dong

Shingu Jun

Patrick F Proppe

Md Tanvir Islam Aumi

Input indicative of a selection of volumetric content for presentation is received. The volumetric content comprises a volumetric representation of one or more elements of a real-world three-dimensional space. In response to the input, device state data associated with the volumetric content is accessed. The device state data describes a state of one or more network-connected devices associated with the real-world three-dimensional space. The volumetric content is presented. The presentation of the volumetric content includes presentation of the volumetric representation of the one or more elements overlaid on the real-world three-dimensional space by a display device and configuring the one or more network-connected devices using the device state data.

Multisensorial presentation of volumetric content

Systems and methods are described for sending a snapshot message using an electronic eyewear device by a first user capturing an image, identifying a physical marker in the captured image, and determining that the physical marker in the captured image has been within a field of view of the electronic eyewear device for a predetermined amount of time. A 3D snapshot including an object is captured and, each time the identified object appears in a field of view of the electronic eyewear device, a representation of the object is sent for display to a second user as if the real object was sent. The representation of the object may include the snapshot image, the object extracted from the snapshot image, a stored representation of the object, augmented reality content representing the object, etc.

Snapshot messages for indicating user state

A volumetric content presentation system includes a head-worn display device, which includes one or more processors, and a memory storing instructions that, when executed by the one or more processors, configure the display device to access AR content items that correspond to either real-world objects or virtual objects, mix and match these AR content items, and present volumetric content that includes these mixed and matched AR content items overlaid on a real-world environment to create a new AR scene that a user can experience.

Mixing and matching volumetric contents for new augmented reality experiences

Augmented reality (AR) systems, devices, media, and methods are described for capturing and presenting effort put into generating a handcrafted AR experience. AR object generation data is captured during the generation of a handcrafted AR object. The AR object generation data is then processed to generate proof of effort data for inclusion with the handcrafted AR object. Examples of proof of effort include a time lapse view of the steps taken during generation of the AR object and statistics such as total time spent, number of images or songs considered for selection, number of actions implemented, etc.

Handcrafted augmented reality effort evidence

Augmented reality (AR) systems, devices, media, and methods are described for generating AR experiences including interactions with virtual or physical prop objects. The AR experiences are generated by capturing images of a scene with a camera system, identifying an object receiving surface and corresponding surface coordinates within the scene, identifying an AR primary object and a prop object (physical or virtual), establishing a logical connection between the AR primary object and the prop object, generating AR overlays including actions associated with the AR primary object responsive to commands received via a user input system that position the AR primary object adjacent the object receiving surface responsive to the primary object coordinates and the surface coordinates within the scene and that position the AR primary object and the prop object with respect to one another in accordance with the logical connection, and presenting the generated AR overlays with a display system.

Augmented reality prop interactions

The present disclosure relates to methods and systems for providing a touch-based augmented reality (AR) experience. During a capture phase, a first user may grip an object. An intensity of a force applied on the object in the grip and/or a duration of the grip may be recorded. A volumetric representation of the first user holding the object may also be captured. During an experience phase, a second user may touch the object, the object may provide haptic feedback (e.g., a vibration) to the second user at an intensity and a duration corresponding to an intensity of the force applied on the object and a duration of the grip of the object. If a volumetric representation of the first user holding the object is captured, touching the object may also cause a presentation of the first user's volumetric body that holds the object.

Touch-based augmented reality experience

A display device presents volumetric content comprising a volumetric video. The volumetric video comprises a volumetric representation of one or more elements a three-dimensional space. Input indicative of a control operation associated with the presentation of the volumetric video is received. The presentation of the volumetric video by the display device is controlled by executing the control operation. While the control operation is being executed, the volumetric representation of the one or more elements of the three-dimensional space are displayed from multiple perspectives based on movement of a user.

Controlling and editing presentation of volumetric content

An olfactory sticker used in chats between electronic devices to make messaging more immersive, personalized, and authentic by integrating olfactory information with traditional text-based formats and AR messaging. Olfactory stickers are used to indicate to a recipient that a message includes olfactory information. The olfactory sticker illustrates a graphical representation of a particular scent that can be sent and received via a chat or an AR message. Olfactory stickers provide the recipient control of accessing the transmitted scent at a desired time. This is particularly useful since certain olfactory information, i.e., scents, can be very direct and intrusive. Olfactory stickers are activated (i.e., release their scent) when they are tapped or rubbed by the recipient.

Olfactory stickers for chat and AR-based messaging

Augmented reality (AR) systems, devices, media, and methods are described for creating a handcrafted AR experience. The handcrafted AR experiences are created by capturing images of a scene, identifying an object receiving surface and corresponding surface coordinates, identifying a customizable AR primary object associated with at least one set of primary object coordinates, generating AR overlays including the customizable AR primary object for positioning adjacent the object receiving surface, presenting the AR overlays, receiving customization commands, generating handcrafted AR overlays including customizations associated with the customizable AR primary object responsive to the customization commands, presenting the handcrafted AR overlays, recording the handcrafted AR overlays, creating a handcrafted AR file including the recorded overlays, and transmitting the handcrafted AR file.

Handcrafted augmented reality experiences

Systems and methods for controlling an Internet of Things (IoT) device through interaction with an augmented reality (AR) object includes pairing an AR object with an IoT device, presenting the AR object on a display of an AR camera device of a user, receiving an interaction signal indicating that the user has interacted with the AR object on the display of the AR camera device, and sending a control signal to the IoT device paired with the AR object in response to the interaction signal. A second user may request presentation of the AR object to the AR display of the AR camera device of the user when the user's AR camera device is located at particular world coordinates. Also, the control signal may be sent when a particular series of interactions with the AR object have been completed, as during game play.

Controlling IoT devices through AR object interaction

A system for interacting with an autonomous vehicle includes a sensor included in the autonomous vehicle and configured to generate sensor data corresponding to a projected hailing area; a projection system included in the autonomous vehicle and configured to generate the projected hailing area on a surface proximate the autonomous vehicle; and a processor included in the autonomous vehicle and configured to execute instructions to: analyze the sensor data to detect the person within the projected hailing area; and in response to detecting the person within the projected hailing area, cause an acknowledgment indicator to be outputted.

Autonomous vehicle interaction system

Systems and methods are described for providing scan based imaging using an electronic eyewear device. The methods include scanning a scene using the electronic eyewear device to capture at least one image in an environment of a first user and identifying at least one physical marker in the scanned scene. Upon identification of the at least one physical marker in the scanned scene, a message (e.g., preselected AR content) is passively sent to at least one of directly to a second user or to at least one physical marker at a remote location for presentation to the second user. The message may be sent without use of the first user's hands to make a selection. The physical markers may be associated with a fixed object, a human face, a pet, a vehicle, a person, a logo, and the like.

Scan-based messaging for electronic eyewear devices

Augmented reality (AR) systems, devices, media, and methods are described for sending and receiving AR objects (e.g., customized AR objects) based on/responsive to interactions with the physical world. AR virtual delivery route overlays are generated responsive to selected virtual delivery routes and include the AR object and a delivery mode (air, tunnel, etc.) corresponding to the virtual delivery route. Physical world actions associated with the delivery mode (blowing adjacent an AR device or scratching a surface) result in sending a communication corresponding to the AR object for delivery to a receiver and generating AR sending overlays including the AR object moving in accordance with the delivery mode.

Physical action-based augmented reality communication exchanges

Interactive experiences are generated using an augmented reality (AR) system that leverages the native input and output capabilities of AR devices to generate immersive and interactive experiences in smart spaces that combine visually-represented programming logic, AR content, and IoT devices. The system connects IoT devices with each other in a native AR environment, provides virtual content and logic gates, scripts sequences and loops of actions, solves visibility issues for IoT devices that are far apart, and designs logic gates and flow circuits that best support user intentions when generating interactive experiences in smart spaces. The interactive experience is produced by generating a program of behaviors of selected IoT devices using logic gates and control circuits connected to the IoT devices in a visual programming interface of the AR device. Once completed, the program of behaviors is executed by a runtime module to control the IoT devices during the interactive experience.

AR system for providing interactive experiences in smart spaces

Systems and methods are described for selectively sharing audio and video streams amongst electronic eyewear devices. Each electronic eyewear device includes a camera arranged to capture a video stream in an environment of the wearer, a microphone arranged to capture an audio stream in the environment of the wearer, and a display. A processor of each electronic eyewear device executes instructions to establish an always-on session with other electronic eyewear devices and selectively shares an audio stream, a video stream, or both with other electronic eyewear devices in the session. Each electronic eyewear device also generates and receives annotations from other users in the session for display with the selectively shared video stream on the display of the electronic eyewear device that provided the selectively shared video stream. The annotation may include manipulation of an object in the shared video stream or overlay images registered with the shared video stream.

Hyper-connected and synchronized AR glasses

Growing community of inventors

San Jose, CA, United States of America

Sven Kratz