Snap Inc.

John James Robertson

Yu Jiang Tham

Gerald Nilles

Jason Heger

Praveen Babu Vadivelu

Dmitry Ryuma

Antoine Ménard

Tamer El Calamawy

Matthias Kalkgruber

Samuel Ahn

Alexander Sourov

Andrew Joseph Bartow

Matthew Du

Anton Kondratenko

Sumant Milind Hanumante

Daniel Wolf

Bernhard Jung

Bo Ding

Edward Lee Kim-Koon

Alexander Kane

Chintan Doshi

Zsolt Mathe

Andrea Ashwood

Mingyang Chai

Mehmood Zafarullahkhan

A magnification application for use with augmented reality (AR) eyewear devices. The magnification application enables a user of an eyewear device to activate a magnification overlay on a display whenever a camera on the eyewear device detects that the field of view includes a registered virtual marker. The magnified overlay includes one or more frames of the captured video data, presented according to a predefined and configurable magnification power. A pointer including a vector and a visual tether guides the user toward the virtual marker. When the eyewear device location is near a perimeter associated with the virtual marker, the magnified overlay appears in a predefined and configurable frame on the display.

Magnified overlays correlated with virtual markers

An eyewear device having an image processor operable in a camera pipeline for computer vision (CV) and in augmented reality (AR) systems. The image processor is configured to selectively control a plurality of cameras to provide images having a first resolution in the high power AR mode, and to provide the images having a second resolution in the low power CV mode. The first resolution is higher than the second resolution, and the plurality of cameras consume less power in the CV mode than the AR mode. The image processor controls the camera pipeline to process the first resolution high IQ images from the plurality of cameras to operate in the AR mode, and controls the camera pipeline to process the second resolution lower IQ images from the plurality of cameras to operate in the CV mode. Substantial power is saved by reducing the resolution of the images using downscaling in the cameras themselves in the CV mode.

Ultra low power camera pipeline for CV in AR systems

A method and apparatus for monitoring a surface temperature of eyewear proximate a processor to understand the surface temperature as a function of computer instructions, such as when the computer instructions are modified during software design. A sensor is coupled to the eyewear proximate the processor, such as at a temple of the eyewear including the processor, using one or more layers of tape. A server provides instructions to the processor for execution, such as instructions of an application, which instructions vary the utilization of the processor. A testing device, such as a digital multi-meter, is coupled to the sensor, as well as the server, and displays the surface temperature as a function of the processor utilization. The surface temperature of the eyewear is monitored to ensure the surface temperature does not exceed a temperature threshold.

Eyewear surface temperature evaluation

Systems and methods for generating an alert on an eyewear device are provided. The systems and methods include receiving, by an eyewear device, from a mobile device, data indicative of a first combination of notification attributes that trigger a first alert on the eyewear device; determining that the mobile device has received a new notification based on additional data received from the mobile device; determining that a combination of attributes of the new notification matches the first combination of notification attributes; and in response to determining that the combination of the attributes of the new notification matches the first combination of notification attributes, retrieving, from a storage device of the eyewear device, a first visual indicator animation that represents the first alert; and activating a visual indicator of the eyewear device in accordance with the retrieved first visual indicator animation to generate the first alert on the eyewear device.

Eyewear with customizable notifications

Systems, methods, devices, computer readable media, and other various embodiments are described for location management processes in wearable electronic devices. Performance of such devices is improved with reduced time to first fix of location operations in conjunction with low-power operations. In one embodiment, low-power circuitry manages high-speed circuitry and location circuitry to provide location assistance data from the high-speed circuitry to the low-power circuitry automatically on initiation of location fix operations as the high-speed circuitry and location circuitry are booted from low-power states. In some embodiments, the high-speed circuitry is returned to a low-power state prior to completion of a location fix and after capture of content associated with initiation of the location fix. In some embodiments, high-speed circuitry is booted after completion of a location fix to update location data associated with content.

Wearable device location systems

Systems, methods, devices, computer readable media, and other various embodiments are described for location management processes in wearable electronic devices. One embodiment involves pairing a client device with a wearable device, capturing a first client location fix at a first time using the first application and location circuitry of the client device. The client device then receives content from the wearable device, where the content is associated with a content capture time and location state data. The client device then updates a location based on the available data to reconcile the different sets of location data. In some embodiments, additional sensor data, such as data from an accelerometer, is used is used to determine which location data is more accurate for certain content.

Wearable device location accuracy systems

Devices and methods for dynamic power configuration (e.g., reduction) for thermal management (e.g., mitigation) in a wearable electronic device such as an eyewear device. The wearable electronic device monitors its temperature and, responsive to the temperature, configures the services it provides to operate in different modes for thermal mitigation (e.g., to prevent overheating). For example, based on temperature, the wearable electronic device adjusts sensors (e.g., turns cameras on or off, changes the sampling rate, or a combination thereof) and adjusts display components (e.g., adjusted rate at which a graphical processing unit generates images and a visual display is updated). This enables the wearable electronic device to consume less power when temperatures are too high in order to provide thermal mitigation.

Eyewear device dynamic power configuration

Eyewear having an image signal processor (ISP) dynamically operable in a camera pipeline for augmented reality (AR) and computer vision (CV) systems. Multi-purpose cameras are used for simultaneous image capture and CV on wearable AR devices. The cameras are coupled to a frame and configured to generate images, wherein the cameras and the ISP are configured to operate in a first AR mode and capture images having a first resolution suitable for use in AR, and are configured to operate in a second CV mode to provide the images having a second resolution suitable for use in CV. The first resolution in the AR mode is higher than the second resolution in the CV mode, and the cameras and the ISP consume less power in the second CV mode than the first AR mode. The cameras and the ISP save significant system power by operating in the low power mode CV mode.

Multi-purpose cameras for simultaneous capture and CV on wearable AR devices

Wearable device location systems architecture

Devices and methods for dynamic power configuration (e.g., reduction) for thermal management (e.g., mitigation) in a wearable electronic device such as an eyewear device. The wearable electronic device monitors its temperature and, responsive to the temperature, configures the services is provides to operate in different modes for thermal mitigation (e.g., to prevent overheating). For example, based on temperature, the wearable electronic device adjusts sensors (e.g., turns cameras on or off, changes the sampling rate, or a combination thereof) and adjusts display components (e.g., adjusted rate at which a graphical processing unit generates images and a visual display is updated). This enables the wearable electronic device to consume less power when temperatures are too high in order to provide thermal mitigation.

A neural network system includes an eyewear device. The eyewear device has a movement tracker, such as an accelerometer, gyroscope, or an inertial measurement unit for measuring acceleration and rotation. The neural network system tracks, via the movement tracker, movement of the eyewear device from at least one finger contact inputted from a user on an input surface. The neural network system identifies a finger gesture by detecting at least one detected touch event based on variation of the tracked movement of the eyewear device over a time period. The neural network system adjusts the image presented on the image display of the eyewear device based on the identified finger gesture. The neural network system can also detect whether the user is wearing the eyewear device and identify an activity of the user wearing the eyewear device based on the variation of the tracked movement over the time period.

Growing community of inventors

Los Angeles, CA, United States of America

John James Robertson