Snap Inc.

Harman International Industries, Inc.

Fuji Xerox Co., Ltd.

Sven Kratz

Stefan J Marti

Joseph Verbeke

Andrés Monroy-Hernández

Rajan Vaish

Yu Jiang Tham

Adam Boulanger

Brian Anthony Smith

Qiong Liu

Patrick Chiu

Fannie Liu

Lei Zhang

Shang Ma

Neeka Mansourian

Tianying Chen

Timothy Chong

Olivia Seow

Donald G Kimber

Priya Seshadri

Evgeny Burmistrov

Ana Maria Cardenas Gasca

Joanne Leong

Xingyu Liu

Hanseul Jun

Anthony Dunnigan

Maribeth Joy Back

James Vaughan

Ilteris Kaan Canberk

Davide Di Censo

Karen Stolzenberg

Laurent Denoue

David Meisenholder

Shin Hwun Kang

Jun Shingu

Daniel Avrahami

Ryota Mizutani

Yusuke Yamaura

Miteshkumar Patel

Tsung-Yu Tsai

Bee Yian Liew

Hao Tang

Joshua Sweers

Mingwen Dong

Shingu Jun

Patrick F Proppe

Md Tanvir Islam Aumi

Andrés Monroy-HERNÁNDEZ

Embodiments of the present disclosure sets forth a computer-implemented method comprising receiving, from at least one sensor, sensor data associated with an environment, computing, based on the sensor data, a cognitive load associated with a user within the environment, computing, based on the sensor data, an affective load associated with an emotional state of the user, determining, based on both the cognitive load at the affective load, an affective-cognitive load, determining, based on the affective-cognitive load, a user readiness state associated with the user, and causing one or more actions to occur based on the user readiness state.

Affective-cognitive load based digital assistant

A system captures via one or more sensors of a computing device, data of an environment observed by the one or more sensors at a first timeslot, and stores the data in a data store as a first portion of a timelapse memory experience. The system also captures, via the one or more sensors of a computing device, data of the environment observed by the one or more sensors at a second timeslot, and stores the data in a data store as a second portion of the timelapse memory experience. The system additionally associates the timelapse memory experience with a memory experience trigger, wherein the memory experience trigger can initiate a presentation of the timelapse memory experience.

Timelapse re-experiencing system

A two-dimensional element is identified from one or more two-dimensional images. A volumetric content item is generated based on the two-dimensional element identified from the one or more two-dimensional images. A display device presents the volumetric content item overlaid on a real-world environment that is within a field of view of a user of the display device.

Generating immersive augmented reality experiences from existing images and videos

Input indicative of a selection of volumetric content for presentation is received. The volumetric content comprises a volumetric representation of one or more elements of a real-world three-dimensional space. In response to the input, device state data associated with the volumetric content is accessed. The device state data describes a state of one or more network-connected devices associated with the real-world three-dimensional space. The volumetric content is presented. The presentation of the volumetric content includes presentation of the volumetric representation of the one or more elements overlaid on the real-world three-dimensional space by a display device and configuring the one or more network-connected devices using the device state data.

Multisensorial presentation of volumetric content

Systems and methods are described for sending a snapshot message using an electronic eyewear device by a first user capturing an image, identifying a physical marker in the captured image, and determining that the physical marker in the captured image has been within a field of view of the electronic eyewear device for a predetermined amount of time. A 3D snapshot including an object is captured and, each time the identified object appears in a field of view of the electronic eyewear device, a representation of the object is sent for display to a second user as if the real object was sent. The representation of the object may include the snapshot image, the object extracted from the snapshot image, a stored representation of the object, augmented reality content representing the object, etc.

Snapshot messages for indicating user state

A volumetric content presentation system includes a head-worn display device, which includes one or more processors, and a memory storing instructions that, when executed by the one or more processors, configure the display device to access AR content items that correspond to either real-world objects or virtual objects, mix and match these AR content items, and present volumetric content that includes these mixed and matched AR content items overlaid on a real-world environment to create a new AR scene that a user can experience.

Mixing and matching volumetric contents for new augmented reality experiences

Augmented reality (AR) systems, devices, media, and methods are described for capturing and presenting effort put into generating a handcrafted AR experience. AR object generation data is captured during the generation of a handcrafted AR object. The AR object generation data is then processed to generate proof of effort data for inclusion with the handcrafted AR object. Examples of proof of effort include a time lapse view of the steps taken during generation of the AR object and statistics such as total time spent, number of images or songs considered for selection, number of actions implemented, etc.

Handcrafted augmented reality effort evidence

Augmented reality (AR) systems, devices, media, and methods are described for generating AR experiences including interactions with virtual or physical prop objects. The AR experiences are generated by capturing images of a scene with a camera system, identifying an object receiving surface and corresponding surface coordinates within the scene, identifying an AR primary object and a prop object (physical or virtual), establishing a logical connection between the AR primary object and the prop object, generating AR overlays including actions associated with the AR primary object responsive to commands received via a user input system that position the AR primary object adjacent the object receiving surface responsive to the primary object coordinates and the surface coordinates within the scene and that position the AR primary object and the prop object with respect to one another in accordance with the logical connection, and presenting the generated AR overlays with a display system.

Augmented reality prop interactions

The present disclosure relates to methods and systems for providing a touch-based augmented reality (AR) experience. During a capture phase, a first user may grip an object. An intensity of a force applied on the object in the grip and/or a duration of the grip may be recorded. A volumetric representation of the first user holding the object may also be captured. During an experience phase, a second user may touch the object, the object may provide haptic feedback (e.g., a vibration) to the second user at an intensity and a duration corresponding to an intensity of the force applied on the object and a duration of the grip of the object. If a volumetric representation of the first user holding the object is captured, touching the object may also cause a presentation of the first user's volumetric body that holds the object.

Touch-based augmented reality experience

A display device presents volumetric content comprising a volumetric video. The volumetric video comprises a volumetric representation of one or more elements a three-dimensional space. Input indicative of a control operation associated with the presentation of the volumetric video is received. The presentation of the volumetric video by the display device is controlled by executing the control operation. While the control operation is being executed, the volumetric representation of the one or more elements of the three-dimensional space are displayed from multiple perspectives based on movement of a user.

Controlling and editing presentation of volumetric content

An olfactory sticker used in chats between electronic devices to make messaging more immersive, personalized, and authentic by integrating olfactory information with traditional text-based formats and AR messaging. Olfactory stickers are used to indicate to a recipient that a message includes olfactory information. The olfactory sticker illustrates a graphical representation of a particular scent that can be sent and received via a chat or an AR message. Olfactory stickers provide the recipient control of accessing the transmitted scent at a desired time. This is particularly useful since certain olfactory information, i.e., scents, can be very direct and intrusive. Olfactory stickers are activated (i.e., release their scent) when they are tapped or rubbed by the recipient.

Olfactory stickers for chat and AR-based messaging

Augmented reality (AR) systems, devices, media, and methods are described for creating a handcrafted AR experience. The handcrafted AR experiences are created by capturing images of a scene, identifying an object receiving surface and corresponding surface coordinates, identifying a customizable AR primary object associated with at least one set of primary object coordinates, generating AR overlays including the customizable AR primary object for positioning adjacent the object receiving surface, presenting the AR overlays, receiving customization commands, generating handcrafted AR overlays including customizations associated with the customizable AR primary object responsive to the customization commands, presenting the handcrafted AR overlays, recording the handcrafted AR overlays, creating a handcrafted AR file including the recorded overlays, and transmitting the handcrafted AR file.

Handcrafted augmented reality experiences

Systems and methods for controlling an Internet of Things (IoT) device through interaction with an augmented reality (AR) object includes pairing an AR object with an IoT device, presenting the AR object on a display of an AR camera device of a user, receiving an interaction signal indicating that the user has interacted with the AR object on the display of the AR camera device, and sending a control signal to the IoT device paired with the AR object in response to the interaction signal. A second user may request presentation of the AR object to the AR display of the AR camera device of the user when the user's AR camera device is located at particular world coordinates. Also, the control signal may be sent when a particular series of interactions with the AR object have been completed, as during game play.

Controlling IoT devices through AR object interaction

A system for interacting with an autonomous vehicle includes a sensor included in the autonomous vehicle and configured to generate sensor data corresponding to a projected hailing area; a projection system included in the autonomous vehicle and configured to generate the projected hailing area on a surface proximate the autonomous vehicle; and a processor included in the autonomous vehicle and configured to execute instructions to: analyze the sensor data to detect the person within the projected hailing area; and in response to detecting the person within the projected hailing area, cause an acknowledgment indicator to be outputted.

Autonomous vehicle interaction system

Systems and methods are described for providing scan based imaging using an electronic eyewear device. The methods include scanning a scene using the electronic eyewear device to capture at least one image in an environment of a first user and identifying at least one physical marker in the scanned scene. Upon identification of the at least one physical marker in the scanned scene, a message (e.g., preselected AR content) is passively sent to at least one of directly to a second user or to at least one physical marker at a remote location for presentation to the second user. The message may be sent without use of the first user's hands to make a selection. The physical markers may be associated with a fixed object, a human face, a pet, a vehicle, a person, a logo, and the like.

Scan-based messaging for electronic eyewear devices

Growing community of inventors

San Jose, CA, United States of America

Sven Kratz