Dolby Laboratories Licensing Corporation

Haricharan Lakshman

Ajit Ninan

Wenhui Jia

Chun Chi Wan

Chaitanya Atluru

Alexandre Chapiro

William Rozzi

Shane Mario Ruggieri

Shwetha Ram

Domagoj Baricevic

Jasper Chao

A device and method for video rendering. The device includes a memory and an electronic processor. The electronic processor is configured to receive, from a source device, video data including multiple reference viewpoints, determine a target image plane corresponding to a target viewpoint, determine, within the target image plane, one or more target image regions, and determine, for each target image region, a proxy image region larger than the corresponding target image region. The electronic processor is configured to determine, for each target image region, a plurality of reference pixels that fit within the corresponding proxy image region, project, for each target image region, the plurality of reference pixels that fit within the corresponding proxy image region to the target image region, producing a rendered target region from each target image region, and composite one or more of the rendered target regions to create video rendering.

Multi-resolution multi-view video rendering

Bordering pixels delineating a texture hole region are identified in a target image. Depth values of the bordering pixels are automatically clustered into two depth value clusters. A specific estimation direction is selected from multiple candidate estimation directions for a texture hole pixel in a texture hole region. A depth value of the texture hole pixel is estimated by interpolating depth values of two bordering background pixels in the specific estimation direction. The estimated depth value is used to warp the texture hole pixel into a reference view represented by a temporal reference image. A pixel value of the texture hole pixel is predicted based on a reference pixel value of a reference pixel from the reference image to which the texture hole pixel is warped using the estimated depth value.

Temporal hole filling for depth image based video rendering

Peripheral-vision expanded images are streamed to a video streaming client. The peripheral-vision expanded images are generated from source images in reference to view directions of the viewer at respective time points. View direction data is collected and received in real time while the viewer is viewing display images derived from the peripheral-vision expanded images. A second peripheral-vision expanded image is generated from a second source image in reference to a second view direction of the viewer at a second time point. The second peripheral-vision expanded image has a focal-vision image portion covering the second view direction of the viewer and a peripheral-vision image portion outside the focal-vision image portion. The second peripheral-vision expanded image is transmitted to the video streaming client.

Virtual reality cinema-immersive movie watching for headmounted displays

Bordering pixels delineating a texture hole region in an image are identified. Depth values of the bordering pixels are recorded. The depth values are automatically clustered into two depth value clusters with a depth value threshold separating the two depth value clusters. A subset of bordering background pixels is identified in the bordering pixels as those with depth values in one of the two depth value clusters that is declared as a background depth value cluster. The subset of bordering background pixels is used to predict texture hole pixel values in the texture hole region based on multiple candidate prediction directions. Quality indicator values are computed for the multiple candidate prediction directions and used to select a specific candidate prediction direction for filling in final texture hole pixel values in the texture hole region of the image.

Hole filling for depth image based rendering

A target view to a 3D scene depicted by a multiview image is determined. The multiview image comprises multiple sampled views. Each sampled view comprises multiple texture images and multiple depth images in multiple image layers. The target view is used to select, from the multiple sampled views of the multiview image, sampled views. A texture image and a depth image for each sampled view in the selected sampled views are encoded into a multiview video signal to be transmitted to a downstream device.

Coding multiview video

A target view to a 3D scene depicted by a multiview image is determined. The multiview image comprising sampled views at sampled view positions distributed throughout a viewing volume. Each sampled view in the sampled views comprises a wide-field-of-view (WFOV) image and a WFOV depth map as seen from a respective sampled view position in the sampled view positions. The target view is used to select, from the sampled views, a set of sampled views. A display image is caused to be rendered on a display of a wearable device. The display image is generated based on a WFOV image and a WFOV depth map for each sampled view in the set of sampled views.

Growing community of inventors

Sunnyvale, CA, United States of America

Haricharan Lakshman