Google Inc.

Toyota Jidosha Kabushiki Kaisha

David Kim

Shahram Izadi

Danhang Tang

Adarsh Prakash Murthy Kowdle

Jonathan James Taylor

Philip L Davidson

Vladimir Tankovich

Cem Keskin

Dongeek Shin

Sofien Bouaziz

Yusuke Nakano

Eric Martinson

Ivan Poupyrev

Shwetak N Patel

Emrah Akin Sisbot

Alex Olwal

Ruofei Du

Christoph Rhemann

Steven L Goldberg

Tristan Schmelcher

Sean Ryan Fanello

Sameh Khamis

Julien Pascal Christophe Valentin

Mingsong Dou

Mathieu Simon Le Goc

Alexander Kharitonov

Laura McKnight

Markus Kuhn

A method includes receiving, by a processor, from a first server, a first plurality of frames associated with a media item. Each frame of the first plurality of frames is associated with a respective timestamp of a first plurality of timestamps generated by the first server. A second plurality of frames associated with the media item are received from a second server. The second plurality of frames are each associated with a respective timestamp of a second plurality of timestamps generated by the second server. An offset value between a first timestamp of the first plurality of timestamps and a second timestamp of the second plurality of timestamps is determined. A modified plurality of frames is generated by modifying, based on the offset value, each timestamp of a subset of the second plurality of timestamps. The modified plurality of frames is sent to the client device.

Systems and methods for synchronization of independently encoded media streams

Systems and methods are provided in which physical objects in the ambient environment can function as user interface implements in an augmented reality environment. A physical object detected within a field of view of a camera of a computing device may be designated as a user interface implement in response to a user command. User interfaces may be attached to the designated physical object, to provide a tangible user interface implement for user interaction with the augmented reality environment.

Tangible six-degree-of-freedom interfaces for augmented reality

Techniques of identifying gestures include detecting and classifying inner-wrist muscle motions at a user's wrist using micron-resolution radar sensors. For example, a user of an AR system may wear a band around their wrist. When the user makes a gesture to manipulate a virtual object in the AR system as seen in a head-mounted display (HMD), muscles and ligaments in the user's wrist make small movements on the order of 1-3 mm. The band contains a small radar device that has a transmitter and a number of receivers (e.g., three) of electromagnetic (EM) radiation on a chip (e.g., a Soli chip. This radiation reflects off the wrist muscles and ligaments and is received by the receivers on the chip in the band. The received reflected signal, or signal samples, are then sent to processing circuitry for classification to identify the wrist movement as a gesture.

Hand gesture recognition based on detected wrist muscular movements

A computing device, such as a wearable device, may include a gesture sensor that generates a gesture signal in response to a gesture of a user performed while the computing device is being worn or held by the user. A calibration sensor may generate a calibration signal characterizing a degree of tightness with which the computing device is being worn or held by the user. The gesture signal may be calibrated using the calibration signal, to obtain a calibrated gesture signal that is calibrated with respect to the degree of tightness. At least one function of the at least one computing device may be implemented, based on the calibrated gesture signal.

Gesture calibration for devices

An electronic device estimates a pose of a hand by volumetrically deforming a signed distance field using a skinned tetrahedral mesh to locate a local minimum of an energy function, wherein the local minimum corresponds to the hand pose. The electronic device identifies a pose of the hand by fitting an implicit surface model of a hand to the pixels of a depth image that correspond to the hand. The electronic device uses a skinned tetrahedral mesh to warp space from a base pose to a deformed pose to define an articulated signed distance field from which the hand tracking module derives candidate poses of the hand. The electronic device then minimizes an energy function based on the distance of each corresponding pixel to identify the candidate pose that most closely approximates the pose of the hand.

Hand tracking based on articulated distance field

An electronic device estimates a depth map of an environment based on stereo depth images captured by depth cameras having exposure times that are offset from each other in conjunction with illuminators pulsing illumination patterns into the environment. A processor of the electronic device matches small sections of the depth images from the cameras to each other and to corresponding patches of immediately preceding depth images (e.g., a spatio-temporal image patch 'cube'). The processor computes a matching cost for each spatio-temporal image patch cube by converting each spatio-temporal image patch into binary codes and defining a cost function between two stereo image patches as the difference between the binary codes. The processor minimizes the matching cost to generate a disparity map, and optimizes the disparity map by rejecting outliers using a decision tree with learned pixel offsets and refining subpixels to generate a depth map of the environment.

System and method for active stereo depth sensing

The novel technology described in this disclosure includes an example method comprising capturing sensor data using one or more sensors describing a particular environment; processing the sensor data using one or more computing devices coupled to the one or more sensors to detect a participant within the environment; determining a location of the participant within the environment; querying a feature database populated with a multiplicity of features extracted from the environment using the location of the participant for one or more features being located proximate the location of the participant; and selecting, using the one or more computing devices, a scene type from among a plurality of predetermined scene types based on association likelihood values describing probabilities of each feature of the one or more features being located within the scene types.

Autonomous human-centric place recognition

This disclosure describes, according to some implementations, a system and method for adapting object handover from robot to human using perceptual affordances. In an example method, upon receiving sensor data describing surroundings and/or operational state of a robot unit from a sensor, the method calculates a probability of a perceptual classification based on the sensor data. The perceptual classification may be one or more of an environment classification, an object classification, a human action classification, or an electro-mechanical state classification. The method further calculates an affordance of the probability of the perceptual classifier using a preference model, determines a handover action based on the affordance, executes the handover action, and updates the preference model based on feedback.

Adapting object handover from robot to human using perceptual affordances

Growing community of inventors

Mountain View, CA, United States of America

David Kim