Columbia University

Snap Inc.

Sony Corporation

Omron Corporation

Mitsubishi Electric Research Laboratories, Inc.

Other

Wisconsin Alumni Research Foundation

Fotonation Limited

Fotonation Cayman Limited

Remotereality Corporation

Cyclo Vision Technologies, Inc.

Shree Kumar Nayar

Gurunandan Krishnan Gorumkonda

Tomoo Mitsunaga

Karl Bayer

Mohit Gupta

Brian Anthony Smith

Changxi Zheng

Jian Wang

Prerna Chikersal

Peter Neil Belhumeur

Yasunobu Hitomi

Mikhail Fridberg

Chang Xiao

Michael Grossberg

Marian Pho

Jinwei Gu

Moshe Ben-Ezra

Ramesh Raskar

Asaf Zomet

Yasuhiro Ohnishi

Neeraj Kumar

Vijay Anand Nagarajan

Kartik Venkataraman

Dan C Lelescu

Amit Kumar Agrawal

Bedabrata Pain

Terrance Edward Boult

Changyin Zhou

Yasumoto Mori

Andreas Udo Velten

Daisuke Iso

Oliver S Cossairt

Daisuke Mitsumoto

Yicheng Wu

Austin Reiter

Yuanzhen Li

Daniel Miau

Kshitiz Garg

Joshua M Gluckman

Yoav Y Schechner

Daniel Sims

Rahul Swaminathan

Arun Asokan Nair

Srinivasa G Narasimhan

Wenzheng Chen

James Korein

Venkata N Peri

Gurunandan Krishnan

Dimitri Bitouk

Bing Zhou

Vishwanath Saragadam Raja Venkata

Eric Breitbach

Minori Noguchi

Alexandros Eleftheriadis

Jacob Andreou

Paul H Dietz

Kensei Jo

Takeshi Kojima

Jong-Il Park

Takashi Sakai

Masaki Suwa

Dimitris Anastassiou

Fumihito Yasuma

Vlad Branzoi

Masatoshi Kimachi

Sujit Kuthirummal

Felipe Gutierrez Barragan

Sergey Trubko

Moon Hyun Lee

Yong Zhao

Jonathan Westhues

Michael Noland

Erich Bruns

Harish Peri

Assaf Zomet

Amruta Karmarkar

Makoto Odamaki

Qi Yin

Masahiro Wantanabe

L Clayton Yaseen, Ii

Shif-Fu Chang

Anton Nikolaev

Chenhan Xu

Riku Arakawa

Li Zhang

Numair Khalil Ullah Khan

Kevin Xie Chen

Alexander C Berg

To Sho

Hideaki Nii

Jay W Summet

Yasuma Fumihito

A finger gesture recognition system is provided. The finger gesture recognition system includes one or more audio sensors and one or more optic sensors. The finger gesture recognition system captures, using the one or more audio sensors, audio signal data of a finger gesture being made by a user, and captures, using the one or more optic sensors, optic signal data of the finger gesture. The finger gesture recognition system recognizes the finger gesture based on the audio signal data and the optic signal data and communicates finger gesture data of the recognized finger gesture to an Augmented Reality/Combined Reality/Virtual Reality (XR) application.

Finger gesture recognition via acoustic-optic sensor fusion

A device that measures a size of a user's face, referred to as face scaling, using a monocular camera. Depth is calculated from sparse feature points. A face mesh is used to improve the estimation accuracy. A processing pipeline detects face features by applying a face landmark detection algorithm to find the important face feature points such as the eyes, nose, and mouth. The processing pipeline estimates feature points depth using depth obtained through image defocus. The processing pipeline further scales the face using an estimated depth of the face features.

Monocular camera defocus face measuring

A pose tracking system is provided. The pose tracking system includes an EMF tracking system having a user-worn head-mounted EMF source and one or more user-worn EMF tracking sensors attached to the wrists of the user. The EMF source is associated with a VIO tracking system such as AR glasses or the like. The pose tracking system determines a pose of the user's head and a ground plane using the VIO tracking system and a pose of the user's hands using the EMF tracking system to determine a full-body pose for the user. Metal interference with the EMF tracking system is minimized using an IMU mounted with the EMF tracking sensors. Long term drift in the IMU and the VIO tracking system are minimized using the EMF tracking system.

Egocentric human body pose tracking

Methods, devices, media, and other embodiments are described for generating, modifying, and outputting pseudorandom animations that can be synchronized to audio data. In one embodiment, a computer animation model made up of comprising one or more control points is accessed by one or more processors, which associate motion patterns with a first control point of the one or more control points, and associate one or more speed harmonics with the first control point. A set of motion states is identify with a motion state for the combinations of possibilities, and a probability value is assigned to each motion state of the set of motion states. The probability value can be used to probabilistically determine a particular motion state to be part of displayed animation for the computer animation model.

Automated dance animation

Shortcut keypad system for electronic communications comprises first apparatus and second apparatus. First apparatus and second apparatus each comprise an input device, a processor and a memory. The input device comprises selectable items including first selectable item. The processor of the first apparatus receives a selection of the first selectable item, and transmits a signal corresponding to the first selectable item to the second apparatus. The processor of the second apparatus receives the signal corresponding to the first selectable item, and causes the input device of the second apparatus to indicate that the signal corresponding to the first selectable item is received. Other embodiments are described herein.

Shortcut keypad for visual electronic communications

Systems and methods are provided that relate to vibrational input elements configured to provide inputs to control an application executed by a mobile computing device. The vibrational input elements may produce distinct vibration patterns that are detectable by a sensor of the mobile computing device. The respective vibration patterns may correspond to one or more actions that may be performed in relation to the application.

Vibrational input elements

An audio response system can generate multimodal messages that can be dynamically updated on viewer's client device based on a type of audio response detected. The audio responses can include keywords or continuum-based signal (e.g., levels of wind noise). A machine learning scheme can be trained to output classification data from the audio response data for content selection and dynamic display updates.

Audio response messages

Method to retrieve media content items using captured parameters associated with a moment starts with a processor detecting an activation of a selectable element. Processor determines parameters associated with the activation including a date and a time that the activation is detected and a location of the selectable element when the activation is detected. Processor stores in a memory of a client device a first record including the parameters. When a request for the first record is received, processor transmits the request to a server for media content items associated with at least one of the parameters of the first record. Media content items are uploaded to the server from client devices communicatively coupled to the server. Processor causes a display of the client device to display the media content items. Other embodiments are described herein.

Selectable element to retrieve media content items

An apparatus to perform fast data access comprises a receiver, a processor, and a memory. The processor receives using the receiver a light signal from a light source. The light signal can be structured to generate a temporal code. The light source is an optical beacon that includes a Light-Emitting Diode (LED). The processor then decodes the light signal to generate a network address, and causes a display of a client device coupled to the apparatus to display information based on the network address. The network address can be a Uniform Resource Locator (URL) address and the information based on the network address includes a webpage associated with the URL. Other embodiments are described herein.

Fast data accessing system using optical beacons

Images of a scene are received. The images represent viewpoints corresponding to the scene. A pixel map of the scene is computed based on the plurality of images. Multi-plane image (MPI) layers from the pixel map are extracted in real-time. The MPI layers are aggregated. The scene is rendered from a novel viewpoint based on the aggregated MPI layers.

High-definition real-time view synthesis

Methods, devices, media, and other embodiments are described for generating pseudorandom animations matched to audio data on a device. In one embodiment a video is generated and output on a display of the device using a computer animation model. Audio is detected from a microphone of the device, and the audio data is processed to determine a set of audio characteristics for the audio data received at the microphone of the device. A first motion state is randomly selected from the plurality of motion states, one or more motion values of the first motion state are generated using the set of audio characteristics, and the video is updated using the one or more motion values with the computer animation model to create an animated action within the video.

Matching audio to a state-space model for pseudorandom animation

Systems and methods are provided for: receiving an image containing a code that has one or more visual qualities that fail to satisfy respective thresholds; applying a trained machine learning model to find a rough location of the code by generating a bounding box and cropping out the portion of the image; applying another trained machine learning model to the portion of the image to estimate key point locations of the code depicted in the portion of the image, aligning the portion of the image that depicts the code based on the estimated key point locations; and decoding, by the other trained machine learning model, the aligned portion of the image that depicts the code.

Long distance QR code decoding

A client device processing received emoji messages using emoji-first messaging. Text messaging is automatically converted to emojis by an emoji-first application so that only emojis are communicated from one client device to another client device. Each client device has a library of emojis that are mapped to words, which libraries are customizable and unique to the users of the client devices, such that the users can communicate secretly in code. Upon receipt of a string of emojis, a user can select the emoji string to convert to text if desired, for a predetermined period of time.

Client device processing received emoji-first messages

A client device, such as a mobile phone or a mobile phone accessory (e.g., phone case), is provided that receives and transmits data (e.g., a social media code) via light wave communication. The light wave communication may comprise structured light (e.g., projected light patterns). The client device may include a lightbox comprised of LEDs located on a back face of the client device.

Low power light wave communication for mobile and wearable devices

Emoji-first messaging where text messaging is automatically converted to emojis by an emoji-first application so that only emojis are communicated from one client device to another client device. Each client device has a library of emojis that are mapped to words, which libraries are customizable and unique to the users of the client devices, such that the users can communicate secretly in code. Upon receipt of a string of emojis, a user can select the emoji string to convert to text if desired, for a predetermined period of time.

Emoji-first messaging

Growing community of inventors

New York, NY, United States of America

Shree Kumar Nayar