Google Inc.

Nicholas Edward Gillian

Jaime Lien

Ivan Poupyrev

Patrick M Amihood

Carsten C Schwesig

Changzhan Gu

Michal Matuszak

Jian Jim Wang

Andrew C Felch

Eiji Hayashi

Abhijit Aroon Shah

Octavio Ponce Madrigal

Abel Seleshi Mengistu

Daniel Lee Giles

Jian Wang

Jin Yamanaka

Blake Charles Jacquot

A system for motion capture of human body movements includes sensor nodes configured for coupling to respective portions of a human subject. Each sensor node generates inertial sensor data as the human subject engages in a physical activity session and processes the inertial sensor data according to a first machine-learned model to generate a set of local motion determinations. One or more computing devices receive the sets of local determinations and process them according to a second machine-learned model to generate a body motion profile. The computing device(s) provide an animated display of an avatar moving according to the body motion profile and generate training data based on input received from a viewer in response to the animated display. The computing device(s) modify at least one of the first machine-learned model or the second machine-learned model based at least in part on the training data.

Dynamic animation of human motion using wearable sensors and machine learning

Techniques and apparatuses are described that facilitate ambient computing using a radar system. Compared to other smart devices that rely on a physical user interface, a smart device with a radar system can support ambient computing by providing an eye-free interaction and less cognitively demanding gesture-based user interface. The radar system can be designed to address a variety of challenges associated with ambient computing, including power consumption, environmental variations, background noise, size, and user privacy. The radar system uses an ambient-computing machine-learned module to quickly recognize gestures performed by a user up to at least two meters away. The ambient-computing machine-learned module is trained to filter background noise and have a sufficiently low false positive rate to enhance the user experience.

Facilitating ambient computing using a radar system

Techniques and apparatuses are described that implement a smart-device-based radar system capable of performing gesture recognition using a space time neural network. The space time neural network employs machine learning to recognize a user's gesture based on complex radar data. The space time neural network is implemented using a multi-stage machine-learning architecture, which enables the radar system to conserve power and recognize the user's gesture in real time (e.g., as the gesture is performed). The space time neural network is also adaptable and can be expanded to recognize multiple types of gestures, such as a swipe gesture and a reach gesture, without significantly increasing size, computational requirements, or latency.

Smart-device-based radar system performing gesture recognition using a space time neural network

This document describes apparatuses and techniques for radar-enabled sensor fusion. In some aspects, a radar field is provided and reflection signals that correspond to a target in the radar field are received. The reflection signals are transformed to provide radar data, from which a radar feature indicating a physical characteristic of the target is extracted. Based on the radar features, a sensor is activated to provide supplemental sensor data associated with the physical characteristic. The radar feature is then augmented with the supplemental sensor data to enhance the radar feature, such as by increasing an accuracy or resolution of the radar feature. By so doing, performance of sensor-based applications, which rely on the enhanced radar features, can be improved.

Radar-enabled sensor fusion

Techniques and apparatuses are described that implement a smart-device-based radar system capable of detecting a frame-of-reference change. In particular, a radar system includes a frame-of-reference machine-learned module trained to recognize whether or not the radar system's frame of reference changes. The frame-of-reference machine-learned module analyzes complex radar data generated from at least one chirp of a reflected radar signal to analyze a relative motion of at least one object over time. By analyzing the complex radar data directly using machine learning, the radar system can operate as a motion sensor without relying on non-radar-based sensors, such as gyroscopes, inertial sensors, or accelerometers. With knowledge of whether the frame-of-reference is stationary or moving, the radar system can determine whether or not a gesture is likely to occur and, in some cases, compensate for the relative motion of the radar system itself.

Detecting a frame-of-reference change in a smart-device-based radar system

Techniques and apparatuses are described that implement a smart-device-based radar system capable of detecting user gestures in the presence of saturation. In particular, a radar system employs machine learning to compensate for distortions resulting from saturation. This enables gesture recognition to be performed while the radar system's receiver is saturated. As such, the radar system can forgo integrating an automatic gain control circuit to prevent the receiver from becoming saturated. Furthermore, the radar system can operate with higher gains to increasing sensitivity without adding additional antennas. By using machine learning, the radar system's dynamic range increases, which enables the radar system to detect a variety of different types of gestures having small or large radar cross sections, and performed at various distances from the radar system.

Saturation compensation using a smart-device-based radar system

Various embodiments wirelessly detect micro gestures using multiple antenna of a gesture sensor device. At times, the gesture sensor device transmits multiple outgoing radio frequency (RF) signals, each outgoing RF signal transmitted via a respective antenna of the gesture sensor device. The outgoing RF signals are configured to help capture information that can be used to identify micro-gestures performed by a hand. The gesture sensor device captures incoming RF signals generated by the outgoing RF signals reflecting off of the hand, and then analyzes the incoming RF signals to identify the micro-gesture.

Gesture recognition using multiple antenna

Techniques and apparatuses are described that implement a smart-device-based radar system capable of detecting user gestures in the presence of saturation. In particular, a radar systememploys machine learning to compensate for distortions resulting from saturation. This enables gesture recognition to be performed while the radar system's receiveris saturated. As such, the radar systemcan forgo integrating an automatic gain control circuit to prevent the receiverfrom becoming saturated. Furthermore, the radar systemcan operate with higher gains to increasing sensitivity without adding additional antennas. By using machine learning, the radar system's dynamic range increases, which enables the radar systemto detect a variety of different types of gestures having small or large radar cross sections, and performed at various distances from the radar system

Smart-device-based radar system detecting user gestures in the presence of saturation

Techniques and apparatuses are described that implement a smart-device-based radar system capable of performing angular estimation using machine learning. In particular, a radar systemincludes an angle-estimation modulethat employs machine learning to estimate an angular position of one or more objects (e.g., users). By analyzing an irregular shape of the radar system's spatial response across a wide field of view, the angle-estimation modulecan resolve angular ambiguities that may be present based on the angle to the object or based on a design of the radar systemto correctly identify the angular position of the object. Using machine-learning techniques, the radar systemcan achieve a high probability of detection and a low false-alarm rate for a variety of different antenna element spacings and frequencies.

Smart-device-based radar system performing angular estimation using machine learning

Various embodiments dynamically learn user-customizable input gestures. A user can transition a radar-based gesture detection system into a gesture-learning mode. In turn, the radar-based gesture detection system emits a radar field configured to detect a gesture new to the radar-based gesture detection system. The radar-based gesture detection system receives incoming radio frequency (RF) signals generated by the outgoing RF signal reflecting off the gesture, and analyzes the incoming RF signals to learn one or more identifying characteristics about the gesture. Upon learning the identifying characteristics, the radar-based gesture detection system reconfigures a corresponding input identification system to detect the gesture when the one or more identifying characteristics are next identified, and transitions out of the gesture-learning mode.

Growing community of inventors

Palo Alto, CA, United States of America

Nicholas Edward Gillian