Google Inc.

James Fung

Joel Hesch

Johnny Hsienchow Lee

Ryan Michael Hickman

Johnny Chung Lee

Ashish A Shah

Patrick Mihelich

A head mounted display (HMD) adjusts feature tracking parameters based on a power mode of the HMD. Examples of feature tracking parameters that can be adjusted include the number of features identified from captured images, the scale of features identified from captured images, the number of images employed for feature tracking, and the like. By adjusting its feature tracking parameters based on its power mode, the HMD can initiate the feature tracking process in low-power modes and thereby shorted the time for high-fidelity feature tracking when a user initiates a VR or AR experience at the HMD.

Low-power mode feature identification at a head mounted display

An electronic device includes at least one sensor, a display, and a processor. The processor is configured to determine a dimension of a physical object along an axis based on a change in position of the electronic device when the electronic device is moved from a first end of the physical object along the axis to a second end of the physical object along the axis. A method includes capturing and displaying imagery of a physical object at an electronic device, and receiving user input identifying at least two points of the physical object in the displayed imagery. The method further includes determining, at the electronic device, at least one dimensional aspect of the physical object based on the at least two points of the physical object using a three-dimensional mapping of the physical object.

Measuring dimension of object through visual odometry

An electronic device balances gain and exposure at an imaging sensor of the device based on detected image capture conditions, such as motion of the electronic device, distance of a scene from the electronic device, and predicted illumination conditions for the electronic device. By balancing the gain and exposure, the quality of images captured by the imaging sensor is enhanced, which in turn provides for improved support of location-based functionality.

Balancing exposure and gain at an electronic device based on device motion and scene distance

Methods and systems for acquiring sensor data using multiple acquisition modes are described. An example method involves receiving, by a co-processor and from an application processor, a request for sensor data. The request identifies at least two sensors of a plurality of sensors for which data is requested. The at least two sensors are configured to acquire sensor data in a plurality of acquisition modes, and the request further identifies for the at least two sensors respective acquisition modes for acquiring data that are selected from among the plurality of acquisition modes. In response to receiving the request, the co-processor causes the at least two sensors to acquire data in the respective acquisition modes. The co-processor receives first sensor data from a first sensor and second sensor data from a second sensor, and the co-processor provides the first sensor data and the second sensor data to the application processor.

Methods and systems for acquiring sensor data on a device using multiple acquisition modes

Methods and systems for providing depth data and image data to an application processor on a mobile device are described. An example method involves receiving image data from at least one camera of the mobile device and receiving depth data from a depth processor of the mobile device. The method further involves generating a digital image that includes at least the image data and the depth data. The depth data may be embedded in pixels of the digital image, for instance. Further, the method then involves providing the digital image to an application processor of the mobile device using a camera bus interface. Thus, the depth data and the image data may be provided to the application processor in a single data structure.

Methods and systems for providing sensor data and image data to an application processor in a digital image format

Methods and systems for communicating sensor data on a mobile device are described. An example method involves receiving, by a processor and from an inertial measurement unit (IMU), sensor data corresponding to a first timeframe, and storing the sensor data using a data buffer. The processor may also receive image data and sensor data corresponding to a second timeframe. The processor may then generate a digital image that includes at least the image data corresponding to the second timeframe and the sensor data corresponding to the first timeframe and the second timeframe. The processor may embed the stored sensor data corresponding to the first timeframe and the second timeframe in pixels of the digital image. And the processor may provide the digital image to an application processor of the mobile device.

Methods and systems for communicating sensor data on a mobile device

Methods and systems for detecting frame tears are described. As one example, a mobile device may include at least one camera, a sensor, a co-processor, and an application processor. The co-processor is configured to generate a digital image including image data from the at least one camera and sensor data from the sensor. The co-processor is further configured to embed a frame identifier corresponding to the digital image at least two corner pixels of the digital image. The application processor is configured to receive the digital image from the co-processor, determine a first value embedded in a first corner pixel of the digital image, and determined a second value embedded in a second corner pixel of the digital image. The application processor is also configured to provide an output indicative of a validity of the digital image based on a comparison between the first value and the second value.

Methods and systems for detecting frame tears

Methods and systems for providing sensor data and image data to an application processor on a mobile device are described. An example method involves receiving image data from at least one camera of the mobile device and receiving sensor data from an inertial measurement (IMU) of the mobile device. The method further involves generating a digital image that includes at least the image data and the sensor data. The sensor data may be embedded in pixels of the digital image, for instance. Further, the method then involves providing the digital image to an application processor of the mobile device using a camera bus interface. Thus, the sensor data and the image data may be provided to the application processor in a single data structure.

Methods and systems for cross-validating sensor data are described. An example method involves receiving image data and first timing information associated with the image data, and receiving sensor data and second timing information associated with the sensor data. The method further involves determining a first estimation of motion of the mobile device based on the image data and the first timing information, and determining a second estimation of the motion of the mobile device based on the sensor data and the second timing information. Additionally, the method involves determining whether the first estimation is within a threshold variance of the second estimation. The method then involves providing an output indicative of a validity of the first timing information and the second timing information based on whether the first estimation is within the threshold variance of the second estimation.

Growing community of inventors

Mountain View, CA, United States of America

James Fung