Snap Inc.

Snapchat, Inc.

David Meisenholder

Ilteris Kaan Canberk

Matthew Loukas Hanover

Qiaokun Huang

Lauryn Morris

Jonathan M Rodriguez, Ii

Daniel Moreno

Karen Stolzenberg

Steven Horowitz

Celia Nicole Mourkogiannis

Kyle Goodrich

Kaveh Anvaripour

Donald Giovannini

Mathieu Emmanuel Vignau

Joseph Timothy Fortier

Kenneth Kubala

Justin Huang

Samuel Bryson Thompson

David G Fliszar

Amit Singh

Robert Matthew Bates

Adam Douglas Greengard

Brandon Carrillo

Evan Spiegel

Shin Hwun Kang

Omer Cansizoglu

Simon Saito Haagen Nielsen

Ivan Golub

Sana Park

Faisal Alqadi

Hwan Moon Lee

Tianyi Sun

Xiaorong Li

Nicolai Darmancev

Quinton Roberts

Yihuan Zhou

Yu Jiang Tham

Rajan Vaish

Sven Kratz

Dhritiman Sagar

Brian Anthony Smith

Sumant Milind Hanumante

Benjamin Lucas

Kameron Sheffield

James Powderly

Jonathan Geddes

Viktoria Hwang

Andrei Rybin

Justin Wilder

Raymond Zeng

Augmented reality features are selected for presentation to a display of an electronic eyewear device by using a camera of the electronic eyewear device to capture a scan image and processing the scan image to extract contextual signals. Simultaneously, voice data from the user is captured by a microphone of the electronic eyewear device and voice-to-text conversion of the captured voice data is performed to identify keywords in the voice data. The extracted contextual signals and the identified keywords are then used to select at least one augmented reality feature that matches the extracted contextual signals and the identified keywords, and the selected augmented reality feature is presented to the display for user selection. The contextual information thus refines the search results to provide the augmented reality feature best suited for the context of the scan image captured by the electronic eyewear device.

Contextual visual and voice search from electronic eyewear device

Apparatuses, systems and methods for wearable devices such as eyewear are described. According to one embodiment, the wearable device includes a body, electronics, and a connector. The body is configured to hold one or more optical elements, the body being disposable between a collapsed condition and a wearable condition in which the device is wearable by a user to hold the one or more optical elements within user view. The electronics are carried by the body. The connector is configured to enable establishment of an electrical and/or electronic connection with the electronics via the connector, the connector being housed by the body such that the connector is substantially obscured from view when the body is in the wearable condition, and such that the connector is exposed for connective access when the body is in the collapsed condition.

Eyewear having selectively exposable feature

Systems, methods, and computer readable media for selecting a tilt angle of an augmented reality (AR) display of an AR wearable device. Some examples of the present disclosure capture simulation data of gaze fixations while users are performing tasks using applications resident on the AR wearable device. The tilt angle of the AR display is selected based on including more gaze fixations that are within the field of view (FOV) of the AR display than are outside the FOV of the AR display. In some examples, an AR wearable device is manufactured with a fixed vertical tilt angle for the AR display. In some examples, the AR wearable device can dynamically adjust the vertical tilt angle of the AR display based on the applications that a user of the AR wearable device is likely to use or is using.

Selecting a tilt angle of an AR display

In a camera-enabled electronic device, photo capture is triggered by a press-and-hold input only if the holding duration of the press-and-hold input is greater than a predefined threshold duration. A press-and-hold input shorter in duration than the threshold triggers video capture. Thus, a short press triggers video capture, while a long press triggers photo capture.

Image-capture control

The subject technology receives, by a client device from a storage device, first image data captured by the client device at a previous time. The subject technology receives first metadata corresponding to at least a first image processing operation and a second image processing operation. The subject technology generates, in a first render pass, second image data based on the first metadata and the image processing operation performed on the first image data. The subject technology generates, in a second render pass, third image data based on the first metadata and the second image processing operation performed on the second image data. The subject technology generates second metadata comprising information corresponding to the third image data. The subject technology generates a composite AR content item comprising the second metadata, the third image data, and the first image data.

Post-capture editing of augmented reality content

System, method, and non-transitory computer readable medium for presenting images on a mobile device. Images are presented by monitoring the location and the orientation of the mobile device, requesting previously captured images corresponding to where the previously captured images were captured in relation to the location from a server, receiving the requested previously captured images from the server, selecting images from the requested previously captured images responsive to the location and the orientation of the mobile device, generating overlay images from the selected images including image icons associated with the selected images, presenting the overlay images on an optical assembly, receiving an image selection identifying one of the image icons in the presented overlay images, and display the selected image associated with the identified image icon on the viewing area of the optical assembly.

Geospatial image surfacing and selection

A method for controlling movement of a drone is disclosed. A spatial vector between a flight-capable drone and a reference object is computed. The spatial vector defines a direction and a distance by which the drone is spaced from the reference object. Flightpath attributes based on the computed vector are determined. The flightpath attributes include one or more of a flight direction, a flight distance, and a flight speed. The flight direction is variable as a function of the direction of the spatial vector. The flight distance is variable as a function of the distance of the spatial vector. The flight speed is variable as a function of the distance of the spatial vector. In an automated operation, movement of the drone is controlled according to the determined flightpath attributes.

Spatial vector-based drone control

Disclosed is a method of receiving and processing content-sending inputs received by a head-worn device system including one or more display devices, one or more cameras and a vertically-arranged touchpad. The method includes displaying a content item on the one or more display devices, receiving a touch input on the touchpad corresponding to a send instruction, displaying a carousel of potential recipients, receiving a horizontal touch input on the touchpad, scrolling the carousel left or right on the one or more display devices in response to the horizontal touch input, receiving a tap touch input on the touchpad to select a particular recipient, receiving a further touch input, and in response to the further touch input, transmitting the content item to the selected recipient.

Selecting items displayed by a head-worn display device

Example systems, devices, media, and methods are described for controlling virtual elements or graphical elements on a display in response to hand gestures detected by an eyewear device that is capturing frames of video data with its camera system. An image processing system detects a series of hand shapes in the video data and determines whether it matches a predefined series of hand gestures. Each predefined series of hand gestures is associated with an action. The system controls movement of the virtual element, relative to the display, in accordance with the associated action. In an example hand shape that includes a thumb sliding along an extended finger, the system establishes a finger scale along the extended finger, calibrates a graphical scale with the finger scale, and controls movement of an interactive graphical element, such as a slider, according to the current thumb position relative to the calibrated graphical scale.

Micro hand gestures for controlling virtual and graphical elements

The present application discloses examples of various apparatuses and systems that can be utilized for augmented reality. According to one example, a wearable device that can optionally comprise: a frame configured for wearing by a user; one or more optical elements mounted on the frame; an array having a plurality of light emitting diodes coupled to the one or more optical elements, wherein the one or more optical elements and the array are mounted within a field of view of the user when the frame is worn by the user; and additional onboard electronic components carried by the frame including at least a battery that is configured to provide for electrically powered operation of the array.

Wearable device with in-eye display capability

Augmented reality experiences of a user wearing an electronic eyewear device are captured by at least one camera on a frame of the electronic eyewear device, the at least one camera having a field of view that is larger than a field of view of a display of the electronic eyewear device. An augmented reality feature or object is applied to the captured scene. A photo or video of the augmented reality scene is captured and a first portion of the captured photo or video is displayed in the display. The display is adjusted to display a second portion of the captured photo or video with the augmented reality features as the user moves the user's head to view the second portion of the captured photo or video. The captured photo or video may be transferred to another device for viewing the larger field of view augmented reality image.

Extended field-of-view capture of augmented reality experiences

A virtual mailbox is established for a first user based on a virtual mailbox definition specified by the first user. A message directed to the first user is received from a first device associated with a second user. A location within a real-world environment of the first user corresponding to the virtual mailbox is identified. A marker associated with the virtual mailbox is detected within the real-world environment of the first user. Based on detecting the marker, the second device presents the message overlaid on the real-world environment at the location corresponding to the virtual mailbox.

Marker-based virtual mailbox for augmented reality experiences

Disclosed is a method of receiving and processing navigation inputs executed by one or more processors in a head-worn device system including one or more display devices, one or more cameras and a generally vertically-arranged touchpad. The method comprises displaying a first carousel of AR effects icons, receiving a first horizontal input on the touchpad, rotating the first carousel of AR effects icons in response to first horizontal input, receiving a first touch input on the touchpad to select a particular AR effects icon that is in a selection position in the first carousel, displaying a scene viewed by the one or more cameras, the scene being enhanced with AR effects corresponding to the particular AR effects icon, receiving content capture user input, and in response to the content capture user input, capturing a new content item corresponding to the scene.

Touchpad navigation for augmented reality display device

Growing community of inventors

Manhattan Beach, CA, United States of America

David Meisenholder