Snap Inc.

Ebay, Incorporated

Jennica Pounds

Aidan Rogers

Stanislav Pugach

Qi Pan

Brent Mills

Ozi Egri

Brent Michael Barkman

Ashwani Arya

Kang Chung Liu

Daniel Harris

Ulf Oscar Michel Loenngren

Andrei Rybin

Jonathan M Rodriguez, Ii

Kai Zhou

Eitan Pilipski

Georgios Evangelidis

Farid Zare Seisan

Daniel Colascione

Ryan Chan

Jonathan Geddes

Zsolt Robotka

Márton Gergely Kajtár

Shuang Xu

Xiangdong Peng

Elliot Solomon

Aiswarya Sridharan

Liviu Marius Coconu

Ryan Pruden

Rebecca Jean Lee

Jiaxin Shan

Austin Vaday

A multi-System on Chip (SoC) hand-tracking platform is provided. The multi-SoC hand-tracking platform includes a computer vision SoC and one or more application SoCs. The computer vision SoC hosts a hand-tracking input pipeline. The one or more application SoCs host one or more applications that are consumers of input event data generated by the hand-tracking input pipeline. The applications communicate with some components of the hand-tracking input pipeline using a shared-memory buffer and with some of the components of the hand-tracking input pipeline using Inter-Process Communication (IPC) method calls.

Multi-SoC hand-tracking platform

Eyewear providing a visual indicator to a hearing-impaired user indicating a direction of arrival of a sound relative to the eyewear to help the user obtain greater awareness of the surrounding environment. An eyewear optical assembly includes an image display displaying the visual indicator discernable by the user and corresponding to the direction of arrival of the sound, even when a sound source is not viewable through the optical assembly. The image display also displays an image indicative of the sound source. A front portion of an eyewear frame includes a light array, where one or more lights of the light array is illuminated to indicate the direction of arrival and intensity of the sound source. An array of vibrating devices may also be used to indicate the direction of arrival and intensity of the sound source.

Eyewear with direction of sound arrival detection

A method for recognizing sign language using collaborative augmented reality devices is described. In one aspect, a method includes accessing a first image generated by a first augmented reality device and a second image generated by a second augmented reality device, the first image and the second image depicting a hand gesture of a user of the first augmented reality device, synchronizing the first augmented reality device with the second augmented reality device, in response to the synchronizing, distributing one or more processes of a sign language recognition system between the first and second augmented reality devices, collecting results from the one or more processes from the first and second augmented reality devices, and displaying, in near real-time in a first display of the first augmented reality device, text indicating a sign language translation of the hand gesture based on the results.

Sign language interpretation with collaborative agents

Various embodiments provide for a registry for augmented reality (AR) objects, which can provide AR objects to a client device to support various software or hardware applications. For instance, some embodiments provide for an AR object registry that facilitates or enables registration of one or more AR objects in association with one or more locations across a planet.

Surfacing augmented reality objects

A text entry process for an Augmented Reality (AR) system. The AR system detects, using one or more cameras of the AR system, a start text entry gesture made by a user of the AR system. During text entry, the AR system detects, using the one or more cameras, a symbol corresponding to a fingerspelling sign made by the user. The AR system generates entered text data based on the symbol and provides text in a text scene component of an AR overlay provided by the AR system to the user based on the entered text data.

Fingerspelling text entry

Electronic eyewear device providing simplified audio source separation, also referred to as voice/sound unmixing, using alignment between respective device trajectories. Multiple users of electronic eyewear devices in an environment may simultaneously generate audio signals (e.g., voices/sounds) that are difficult to distinguish from one another. The electronic eyewear device tracks the location of moving remote electronic eyewear devices of other users, or an object of the other users, such as the remote user's face, to provide audio source separation using location of the sound sources. The simplified voice unmixing uses a microphone array of the electronic eyewear device and the known location of the remote user's electronic eyewear device with respect to the user's electronic eyewear device to facilitate audio source separation.

Eyewear with audio source separation using pose trackers

An eyewear device that performs diarization by segmenting spoken language into different speakers and remembering each speaker over the course of a session. The speech of each speaker is translated to text and the text of each speaker is displayed on an eyewear display. The text of each user has a different attribute such that the eyewear user can distinguish the text of different speakers. Examples of the text attribute can be a text color, font, and font size. The text is displayed on the eyewear display such that it does not substantially obstruct the user's vision.

Eyewear including diarization

Augmented reality objects registry

An eyewear having an electronic processor configured to identify a hand gesture including a sign language, and to generate speech that is indicative of the identified hand gesture. The electronic processor uses a convolutional neural network (CNN) to identify the hand gesture by matching the hand gesture in the image to a set of hand gestures, wherein the set of hand gestures is a library of hand gestures stored in a memory. The hand gesture can include a static hand gesture, and a moving hand gesture. The electronic processor is configured to identify a word from a series of hand gestures.

Eyewear including sign language to speech translation

Improved access of a multidimensional database can be implemented using a space-filling curve, such as a z-order curve, and a tree structure. A state engine is configured to track the state of the dimensions of the database as the tree structure is traversed. Different states allow optimizations such as avoidance of analyzing a dimension for further nodes of the tree and removal of multiple nodes in a sub-tree.

Database access using a z-curve

Aspects of the present disclosure include a system comprising a machine-readable storage medium storing instructions and computer-implemented methods for consistency mitigation in real-time streams. Consistent with some embodiments, the method includes reading a message object from a message queue and adding the message object to an aggregating cache. The aggregating cache includes a linked list of ordered message objects. The method further includes applying an aggregating function to the message objects in the linked list and persisting the aggregated message objects to a database. The method further includes clearing the message objects from the aggregating cache based on the message objects being persisted to the database.

Consistency mitigation techniques for real-time streams

Growing community of inventors

Bellevue, WA, United States of America

Jennica Pounds