Snap Inc.

Yu Jiang Tham

Andrés Monroy-Hernández

Rajan Vaish

Jonathan M Rodriguez, Ii

Gerald Nilles

Sven Kratz

Brian Anthony Smith

Nir Daube

Ava Robinson

John James Robertson

Jason Heger

Praveen Babu Vadivelu

Lei Zhang

Ana Maria Cardenas Gasca

Russell Douglas Patton

Fannie Liu

Antoine Ménard

Miran Alhaideri

Stephen Pomes

Youjean Cho

Daekun Kim

Peter Brook

Nicholas Larson

Zhihao Hong

Ilteris Kaan Canberk

Xing Mei

Hemant Surale

Tamer El Calamawy

Tianying Chen

Timothy Chong

Olivia Seow

Simon Nielsen

Jonathan Rodriguez

Xingyu Liu

Hanseul Jun

Joanne Leong

Mitchell Bechtold

Alex Bamberger

Tsung-Yu Tsai

Chunjong Park

Ella Dagan Peled

Karen Stolzenberg

Matthew Loukas Hanover

David Meisenholder

Shin Hwun Kang

Michael Xu

Savanah Frisk

Quan Thoi Minh Nguyen

Andrés Monroy-HERNÁNDEZ

Jonathan M Ii Rodriguez

Michael Jing Xu

Users of electronic eyewear devices can interact with each other by sharing 3D objects (e.g., 2D or 3D augmented reality (AR) objects or scanned 2D or 3D images of real-world objects) with each other via local objects (real or virtual) in each user's environment established as personalized anchor points for social connection. When a user receives an object from another user, the user has the option to generate a connected session with other users that are co-located (physically or virtually at the same location) with the user. The co-located group of users in this new connected session may view the received object either on their personal electronic devices (e.g., smartphones) or on their electronic eyewear devices and can modify and annotate the shared object using collaboration software and AR display tools that enable modification and manipulation of the shared object.

Sharing received objects with co-located users

A system and associated methods display on a user device a non-fungible token (NFT) in an application that displays video data. The methods include storing the NFT of a user in a database accessible to the application, associating an ID of the user for the application to the NFT, and receiving a selection of the NFT during use of the application. Upon selection, the selected NFT is provided to the user device for superimposing the selected NFT over the user's face, providing the selected NFT as a background image for the video data on the user device, or both. The user device includes a web application that connects the database to a crypto wallet containing the NFT. The NFT may be processed to crop out the eyes and mouth of the NFT for registration with the user's eyes and mouth when the NFT is superimposed over the user's face.

User assumption of identity of NFT in crypto wallet

Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, method, and user interface to facilitate augmented reality based communication between multiple users over a network. Session configuration data including configuration parameters of a virtual interaction session with a first user is received from a first device. The configuration parameters include an identifier of a second user that is permitted to join the virtual interaction session and a micro-chat duration that defines a time limit for a real-time communication link between the first and second user during the virtual interaction session. The real-time communication link between the first and second user by causing display, by the second device, of a live camera feed generated at the first device. Upon expiration of the micro-chat duration, the real-time communication link between the first and second user is terminated.

Virtual interaction session to facilitate real-time communication between multiple users

A method and a system include receiving a first signal value generated by a biosignal sensor coupled to the first client device, receiving one or more second signal values corresponding to a respective sensor reading or environmental condition associated with the first client device, determining a total score based on a first score and one or more second scores determined based on the first and the one or more second signal values, selecting a first state of the plurality of states based on a ranking of total scores of the plurality of states, and causing a display of a first notification associated with a first user-selectable element corresponding to the first state on the first client device.

Non-textual communication and user states management

The users' experience of engaging with augmented reality (AR) technology that permits users to interact with their environment and with each other is enhanced by automatically selecting an AR experience that is suitable for use given the physical environment of the user. The physical environment of the user is the physical environment of the user's computing device. The physical environment may include objects and/or conditions present in close proximity to the user's computing device, such as other humans, animals, and smart devices.

Context-based selection of augmented reality experiences

Described are virtual AR interfaces for generating a virtual rotational interface for the purpose of controlling connected IoT devices using the inertial measurement unit (IMU) of a portable electronic device. The IMU control application enables a user of a portable electronic device to activate a virtual rotational interface overlay on a display and adjust a feature of a connected IoT product by rotating a portable electronic device. The device IMU moves a slider on the virtual rotational interface. The IMU control application sends a control signal to the IoT product which executes an action in accordance with the slider position. The virtual rotational interface is presented on the display as a virtual object in an AR environment. The IMU control application detects the device orientation (in the physical environment) and in response presents a corresponding slider element on the virtual rotational interface (in the AR environment).

Virtual AR interfaces for controlling IoT devices using mobile device orientation sensors

A system and method for viewing a virtual digital object created from components having cryptocurrency value includes receiving a selection of at least one of an avatar, a captured image of the user or a third party, or a digital scene and receiving a selection of the virtual digital object for display on the selected avatar or captured image or within the digital scene. The virtual digital object includes metadata relating to at least one of cryptocurrency value of the virtual digital object, creation of the virtual digital object, and rules governing use and ownership of the virtual digital object. The user may manipulate the displayed virtual digital object within the display and may elect to display at least part of the metadata (e.g., price) along with the virtual digital object. The display of the virtual digital object may be captured and forwarded via a messaging or social media application.

Method, system, and computer readable medium for displaying virtual digital objects generated using blockchain technology

Systems and methods are described for sending a snapshot message using an electronic eyewear device by a first user capturing an image, identifying a physical marker in the captured image, and determining that the physical marker in the captured image has been within a field of view of the electronic eyewear device for a predetermined amount of time. A 3D snapshot including an object is captured and, each time the identified object appears in a field of view of the electronic eyewear device, a representation of the object is sent for display to a second user as if the real object was sent. The representation of the object may include the snapshot image, the object extracted from the snapshot image, a stored representation of the object, augmented reality content representing the object, etc.

Snapshot messages for indicating user state

Systems, devices, media, and methods are presented for selectively activating and suspending control of a graphical user interface by two or more electronic devices. A portable eyewear device includes a display projected onto at least one lens assembly and a primary touchpad through which the user may access a graphical user interface (GUI) on the display. A handheld accessory device, such as a ring, includes an auxiliary touchpad that is configured to emulate the primary touchpad. The eyewear processor temporarily suspends inputs from one touchpad when it detects an activation signal from the other touchpad.

Selectively activating a handheld device to control a user interface displayed by a wearable device

Augmented reality (AR) systems, devices, media, and methods are described for capturing and presenting effort put into generating a handcrafted AR experience. AR object generation data is captured during the generation of a handcrafted AR object. The AR object generation data is then processed to generate proof of effort data for inclusion with the handcrafted AR object. Examples of proof of effort include a time lapse view of the steps taken during generation of the AR object and statistics such as total time spent, number of images or songs considered for selection, number of actions implemented, etc.

Handcrafted augmented reality effort evidence

Systems and methods herein describe generating automatic reactions in an augmented reality messenger system. The claimed systems and methods generate a contextual trigger defining a set of conditions at a first computing device, detect at least one of the set of conditions has been satisfied, cause presentation of an augmented reality content item at a second computing device, generate a user reaction in response to the presentation of the augmented reality content item and transmit the user reaction to a first computing device.

Augmented reality auto reactions

Augmented reality (AR) systems, devices, media, and methods are described for generating AR experiences including interactions with virtual or physical prop objects. The AR experiences are generated by capturing images of a scene with a camera system, identifying an object receiving surface and corresponding surface coordinates within the scene, identifying an AR primary object and a prop object (physical or virtual), establishing a logical connection between the AR primary object and the prop object, generating AR overlays including actions associated with the AR primary object responsive to commands received via a user input system that position the AR primary object adjacent the object receiving surface responsive to the primary object coordinates and the surface coordinates within the scene and that position the AR primary object and the prop object with respect to one another in accordance with the logical connection, and presenting the generated AR overlays with a display system.

Augmented reality prop interactions

A method for detecting full-body gestures by a mobile device includes a host mobile device detecting the tracked body of a co-located participant in a multi-party session. When the participant's tracked body provides a full-body gesture, the host's mobile device recognizes that there is a tracked body providing a full-body gesture. The host mobile device iterates through the list of participants in the multi-party session and finds the closest participant mobile device with respect to the screen-space position of the head of the gesturing participant. The host mobile device then obtains the user ID of the closest participant mobile device and broadcasts the recognized full-body gesture event to all co-located participants in the multi-party session, along with the obtained user ID. Each participant's mobile device may then handle the gesture event as appropriate for the multi-party session. For example, a character or costume may be assigned to a gesturing participant.

Co-located full-body gestures

Context-sensitive remote controls for use with electronic devices (e.g., eyewear device). The electronic device is configured to perform activities (e.g., email, painting, navigation, gaming). The context-sensitive remote control includes a display having a display area, a display driver coupled to the display, and a transceiver. The remote control additionally includes memory that stores controller layout configurations for display in the display area of the display by the display driver. A processor in the context-sensitive remote control is configured to establish, via the transceiver, communication with an electronic device, detect an activity currently being performed by the electronic device, select one of the controller layout configurations responsive to the detected activity, and present, via the display driver, the selected controller layout configuration in the display area of the display.

Context-sensitive remote eyewear controller

Eyewear having a light array and vibration sensors for indicating to a user when to turn left and right, such as when the eyewear is operating a navigation map application, that improves the user experience of eyewear devices for users having partial blindness or complete blindness. To compensate for partial blindness, the front portion of the eyewear frame, such as the bridge, may include the light array, where the left portion of the light array is illuminated to indicate to the user to turn left. The right portion of the light array is illuminated to indicate to the user to turn right. To compensate for complete users having complete blindness, the eyewear has a vibration device on each side of the eyewear, such as in the temples, which selectively vibrate to indicate when the user should turn left and right.

Navigation assistance for the visually impaired

Growing community of inventors

Los Angeles, CA, United States of America

Yu Jiang Tham