Google Inc.

Eric Turner

Behnam Bastani

Mark B Spitzer

Adarsh Prakash Murthy Kowdle

Carlin Vieri

Konstantine Nicholas John Tsotsos

Onur Gonen Guleryuz

Jonathan Tilton Barron

Shahram Izadi

Neal Wadhwa

Haomiao Jiang

Chao X Guo

Sean Ryan Francesco Fanello

Christoph Rhemann

Vladimir Tankovich

Ambrus Csaszar

Michael John Schoenberg

Sameh Khamis

Mirko Schmidt

Vivek Verma

Yunwen Zhou

Ivan Dryanovski

Julien Valentin

Maksym Dzitsiuk

Mira Leung

Juan David Hincapie Ramos

Abhishek Kar

Damien Saint-Macary

João Manuel Castro Afonso

Jose Pascoal

Ryan Christopher DuToit

Bicheng Luo

Disclosed is a method including receiving a depth map estimated using data based on image and data received from a movement sensor as input, generating an alignment parameter based on the depth map, adding the alignment parameter to a pre-calibration state to define a user operational calibration state, generating scale parameters and shift parameters based on features associated with the data received from the image and movement sensor, and calibrating the image and movement sensor based on the user operational calibration state, the scale parameters and the shift parameters.

Visual inertial odometry with machine learning depth

A method including receiving (S) a request for a depth map, generating (S) a hybrid depth map based on a device depth map () and downloaded depth information (), and responding (S) to the request for the depth map with the hybrid depth map (). The device depth map () can be depth data captured on a user device () using sensors and/or software. The downloaded depth information () can be associated with depth data, map data, image data, and/or the like stored on a remote (to the user device) server ().

Hybrid depth maps

A handheld user device includes a monocular camera to capture a feed of images of a local scene and a processor to select, from the feed, a keyframe and perform, for a first image from the feed, stereo matching using the first image, the keyframe, and a relative pose based on a pose associated with the first image and a pose associated with the keyframe to generate a sparse disparity map representing disparities between the first image and the keyframe. The processor further is to determine a dense depth map from the disparity map using a bilateral solver algorithm, and process a viewfinder image generated from a second image of the feed with occlusion rendering based on the depth map to incorporate one or more virtual objects into the viewfinder image to generate an AR viewfinder image. Further, the processor is to provide the AR viewfinder image for display.

Depth from motion for augmented reality for handheld user devices

A foveated display system includes a rendering device including at least one graphics processing unit (GPU) to render a foveal region and a peripheral region of a first image, wherein the foveal region has a higher resolution than the peripheral region. The system further includes a display device coupled to the rendering device via at least one physical layer. The display device includes a pixel array and a display controller coupled to the pixel array. The display controller includes a scaling component to upscale the first peripheral region to generate a scaled first peripheral region and a blending component to blend the foveal region with the scaled first peripheral region to generate a second image.

Dual-path foveated graphics pipeline

A display device, such as a head mounted device (HMD), displays a virtual scene. The display device includes a motion tracker for detecting rotation and/or translation of the display device. The display device also includes a processor that is configured to determine, in response to the detected, an orientation of the display device relative to a plurality of world-aligned viewing frustums that are stationary relative to the virtual scene. The processor is also configured to identify a set of those world-aligned viewing frustums that overlap with an output field of view. The processor is further configured to render pixels of the set of those world-aligned viewing frustums that overlap with an output field of view and upsample the rendered pixels to generate values of display pixels for presentation by the display device.

Phase aligned foveated rendering

A display device, such as a head mounted device (HMD), displays a virtual scene. The display device includes a motion tracker for detecting rotation of the display device. The display device also includes a processor that is configured to selectively maintain or modify a position of an array of rendered pixels relative to the virtual scene in response to the detected motion. The processor is also configured to upsample the rendered pixels to generate values of display pixels for presentation by the display device. The processor is further configured to translate the values of the display pixels in a rendering plane of the display device based on the detected motion. The translated values of the display pixels can then be presented on a display of the display device.

A display system includes a display device and a graphics processing unit (GPU) coupled via at least one physical layer. The display device includes a pixel array having a non-red-green-blue (non-RGB) pixel format. The GPU is configured to render an image in the non-RGB pixel format and provide the rendered image for transmission to the pixel array via the at least one physical layer.

Early sub-pixel rendering

Growing community of inventors

Mountain View, CA, United States of America

Eric Turner