Google Inc.

Shiqi Chen

Rahul Garg

Alexander James Faaborg

Jonathan Tompson

Manuel Christian Clement

Johnny Hsienchow Lee

Glen Murphy

Christian Plagemann

Rong Liu

Joel Hesch

Jon M Wiley

Murphy Stein

Andrey Doronichev

Dominik Philemon Kaeser

Clayton Woodward Bavor, Jr

Rohit Kumar Pandey

Zhaoyang Xu

Marie Charisse White

Andrew Nartker

Samuel Mark Matson

Pavlo Pidlypenskyi

Xue Wang

Jon Michael Wiley

Clayton Woodward Bavor

Systems and methods for context-sensitive hand interaction with an immersive environment are provided. An example method includes determining a contextual factor for a user and selecting an interaction mode based on the contextual factor. The example method may also include monitoring a hand of the user to determine a hand property and determining an interaction with an immersive environment based on the interaction mode and the hand property.

Context-sensitive hand interaction

A method for controller tracking with multiple degrees of freedom includes generating depth data at an electronic device based on a local environment proximate the electronic device. A set of positional data is generated for at least one spatial feature associated with a controller based on a pose of the electronic device, as determined using the depth data, relative to the at least one spatial feature associated with the controller. A set of rotational data is received that represents three degrees-of-freedom (3DoF) orientation of the controller within the local environment, and a six degrees-of-freedom (6DoF) position of the controller within the local environment is tracked based on the set of positional data and the set of rotational data.

Controller tracking for multiple degrees of freedom

In at least one aspect, a method can include generating a respective set of training set of images for each label in a handedness model by: receiving the label at an image capturing device, obtaining a set of captured images by recording a pass-through image of a user placing a target object within an overlay of a bounding area animation, the target object corresponding with the label, and associating the label with each image in the set of captured images. The method includes training, using the training images, the handedness model to provide a correct label for an input image.

Methods of determining handedness for virtual controllers

In one aspect, a method and system are described for receiving input for a virtual user in a virtual environment. The input may be based on a plurality of movements performed by a user accessing the virtual environment. Based on the plurality of movements, the method and system can include detecting that at least one portion of the virtual user is within a threshold distance of a collision zone, the collision zone being associated with at least one virtual object. The method and system can also include selecting a collision mode for the virtual user based on the at least one portion and the at least one virtual object and dynamically modifying the virtual user based on the selected collision mode.

Context sensitive hand collisions in virtual reality

A system for detecting and tracking a hover position of a manual pointing device, such as finger(s), on a handheld electronic device may include overlaying a rendered mono-chromatic keying screen, or green screen, on a user interface, such as a keyboard, of the handheld electronic device. A position of the finger(s) relative to the keyboard may be determined based on the detection of the finger(s) on the green screen and a known arrangement of the keyboard. An image of the keyboard and the position of the finger(s) may be rendered and displayed, for example, on a head mounted display, to facilitate user interaction via the keyboard with a virtual immersive experience generated by the head mounted display.

Touchscreen hover detection in an augmented and/or virtual reality environment

In one general aspect, a system for providing a virtual reality (VR) space can include a mobile computing device, and a VR headset operatively coupled to the mobile computing device and including a screen. The mobile computing device can be configured to execute a VR application, and provide content for display on the screen of the VR headset in the VR space.

Virtual reality headset connected to a mobile computing device

A system for tracking a first electronic device, such as a handheld smartphone, in a virtual reality environment generated by a second electronic device, such as a head mounted display may include detection, by a camera included in one of the first electronic device or the second electronic device, of at least one visual marker included on the other of the first electronic device or the second electronic device. Features detected within the field of view corresponding to known features of the visual markers may be used to locate and track movement of the first electronic device relative to the second electronic device, so that movement of the second electronic device may be translated into an interaction in a virtual experience generated by the second electronic device.

System for tracking a handheld device in virtual reality

Growing community of inventors

Mountain View, CA, United States of America

Shiqi Chen