Google Inc.

University of Southern California

Bo Wu

Thad Eugene Starner

Hartmut Neven

Yang Song

Hayes Solos Raffle

Yong Zhao

David Sparks

Andrea Lynn Frome

Michael Patrick Johnson

Alessandro Bissacco

Kong Man Cheung

Hartwig Adam

Sergey Ioffe

Luc Vincent

Mei Han

Edward Allen Keyes

Ahmad A Abdulkader

Raymond W Smith

Shlomo Reuben Urbach

Marco Zennaro

Qixing Huang

Ramakant Nevatia

German Cheung

Nirmal Jitendra Patel

Joshua L Weaver

Sergey Brin

Mat Balez

Yuan Li

Chang Jiang Huang

Alessandro Bissallo

Chao-Yeh Chen

Xiao Zhang

Disclosed are methods and devices for varying functionality of a wearable computing device. An example device includes a first sensor and a second sensor. An example method includes, while a device is operating in a first state, receiving an indication of a touch input at the first sensor. The second sensor is configured in an idle mode based on the device operating in the first state. The method further includes, in response to receiving the indication of the touch input, triggering the second sensor to operate in an active mode and receiving data from the second sensor. The method further includes determining, based on the data, whether the device is being worn.

On-head detection with touch sensing and eye sensing

An image processing system automatically segments and labels an image using a hierarchical classification model. A global classification model determines initial labels for an image based on features of the image. A label-based descriptor is generated based on the initial labels. A local classification model is then selected from a plurality of learned local classification model based on the label-based descriptor. The local classification model is applied to the features of the input image to determined refined labels. The refined labels are stored in association with the input image.

Hierarchical conditional random field model for labeling and segmenting images

Examples of methods and systems for providing calibration for eye gesture recognition are described. In some examples, calibration can be executed via a head-mountable device. A method for calibration of a system may account for changes in orientation of the head-mountable device, update recognition of the eye gestures, or increase efficiency of the system, for example. The head-mountable device may be configured to receive signals indicative of eye gestures from an eye gesture-detection system and in response to receiving a second command confirming that the signal is indicative of an eye gesture command, to make adjustments to the eye gesture recognition system and/or the reference signals. The head-mountable device may calibrate an eye gesture recognition system via implicit or explicit calibration, for example.

Methods and systems for calibrating a device

On-head detection for wearable computing device

A system and method is provided for automatically recognizing building numbers in street level images. In one aspect, a processor selects a street level image that is likely to be near an address of interest. The processor identifies those portions of the image that are visually similar to street numbers, and then extracts the numeric values of the characters displayed in such portions. If an extracted value corresponds with the building number of the address of interest such as being substantially equal to the address of interest, the extracted value and the image portion are displayed to a human operator. The human operator confirms, by looking at the image portion, whether the image portion appears to be a building number that matches the extracted value. If so, the processor stores a value that associates that building number with the street level image.

System and method of determining building numbers

Examples methods and systems for distinguishing winks from other eye motions are described. In some examples, a method for distinguishing winks from other eye motions may account for changes in orientation of a computing device relative to a user, calibrate eye gesture recognition, or increase efficiency of the computing device. A computing device may be configured to receive sensor data corresponding to an eye and determine a distribution of magnitude changes in the sensor data over an interval of time. The computing device may identify clusters that correspond to ranges of magnitude changes within the distribution and use the clusters as reference data to identify ranges of sensor data for different types of eye motions, including differentiating winks from clusters indicative of ranges of normal eye activity and blinks.

Methods and systems for receiving input controls

Methods and systems that are described herein may help to dynamically utilize multiple eye-tracking techniques to more accurately determine eye position and/or eye movement. An exemplary system may be configured to: (a) perform at least a first and a second eye-tracking process; (b) determine a reliability indication for at least one of the eye-tracking processes; (c) determine a respective weight for each of the eye-tracking processes based at least in part on the reliability indication; (d) determine a combined eye position based on a weighted combination of eye-position data from the two or more eye-tracking processes, wherein the eye-position data from each eye-tracking process is weighted by the respectively determined weight for the eye-tracking process; and (e) carry out functions based on the combined eye position.

Input detection

Examples of methods and systems for using eye gesture duration to provide calibration for eye gesture detection are described. In some examples, calibration can be executed using a head-mountable device. The head-mountable device may be configured to determine a duration range indicative of an eye gesture and receive a plurality of reference signals indicative of the eye gesture. The plurality of reference signals may comprise duration information indicative of a plurality of reference durations of the eye gesture. The head-mountable device may determine, based on the plurality of reference durations, a reference duration range associated with the eye gesture that is within the duration range, and adjust the duration range for the eye gesture based on the reference duration range.

Input system

This disclosure involves proximity sensing of eye gestures using a machine-learned model. An illustrative method comprises receiving training data that includes proximity-sensor data. The data is generated by at least one proximity sensor of a head-mountable device (HMD). The data is indicative of light received by the proximity sensor(s). The light is received by the proximity sensor(s) after a reflection of the light from an eye area. The reflection occurs while an eye gesture is being performed at the eye area. The light is generated by at least one light source of the HMD. The method further comprises applying a machine-learning process to the training data to generate at least one classifier for the eye gesture. The method further comprises generating an eye-gesture model that includes the at least one classifier for the eye gesture. The model is applicable to subsequent proximity-sensor data for detection of the eye gesture.

Methods and systems are described for determining eye position and/or for determining eye movement based on glints. An exemplary computer-implemented method involves: (a) causing a camera that is attached to a head-mounted display (HMD) to record a video of the eye; (b) while the video of the eye is being recorded, causing a plurality of light sources that are attached to the HMD and generally directed towards the eye to switch on and off according to a predetermined pattern, wherein the predetermined pattern is such that at least two of the light sources are switched on at any given time while the video of the eye is being recorded; (c) analyzing the video of the eye to detect controlled glints that correspond to the plurality of light sources; and (d) determining a measure of eye position based on the controlled glints.

Position estimation based rotation of switched off light source

This disclosure relates to winking to capture image data using an image capture device that is associated with a head-mountable device (HMD). An illustrative method includes detecting a wink gesture at an HMD. The method also includes causing an image capture device to capture image data, in response to detecting the wink gesture at the HMD.

Image capture technique

This disclosure relates to proximity sensing for wink detection. An illustrative method includes receiving data from a receiver portion of a proximity sensor. The receiver portion is disposed at a side section of a head-mountable device (HMD). When a wearer wears the HMD, the receiver portion is arranged to receive light reflected from an eye area of the wearer, the proximity sensor detects a movement of the eye area, and the data represents the movement. The method includes determining that the data corresponds to a wink gesture. The method also includes selecting a computing action to perform, based on the wink gesture. The method further includes performing the computing action.

Wink gesture input for a head-mountable device

Growing community of inventors

Mountain View, CA, United States of America

Bo Wu