Disney Enterprises, Inc.

Eth Zurich (eidgenoessische Technische Hochschule Zurich)

Eth Zurich

Yael Pritch

Alexander Sorkine Hornung

Henning Zimmer

Markus H Gross

Federico Perazzi

Simon Heinzle

Changil Kim

Philipp Krähenbühl

Lanny Starkes Smoot

Oliver Wang

Quinn Smithwick

Markus Gross

Thomas R Gross

Aljoscha Smolic

Anselm Grundhofer

Alexander Sorkine-Hornung

Fabio Zünd

Nicola V Ranieri

Ulrich Mueller

Christian Richardt

The disclosure provides an approach for estimating depth in a scene. According to one aspect, regions where the depth estimation is expected to perform well may first be identified in full-resolution epipolar-plane images (EPIs) generated from a plurality of images of the scene. Depth estimates for EPI-pixels with high edge confidence are determined by testing a number of discrete depth hypotheses and picking depths that lead to highest color density of sampled EPI-pixels. The depth estimate may also be propagated throughout the EPIs. This process of depth estimation and propagation may be iterated until all EPI-pixels with high edge confidence have been processed, and all EPIs may also be processed in this manner. The EPIs are then iteratively downsampled to coarser resolutions, at which edge confidence for EPI-pixels not yet processed are determined, depth estimates of EPI-pixels with high edge confidence made, and depth estimates propagated throughout the EPIs.

Scene reconstruction from high spatio-angular resolution light fields

Methods and apparatus for deriving a saliency measure for images and video are described. In an embodiment, a process includes decomposing, by a processor, an image into elements, wherein elements cluster the image and each element consists of spatially connected pixels. The processor then calculates a first image measure indicative of each element's uniqueness in the image on a per element basis, and calculates a second image measure indicative of each element's spatial distribution in the image on a per element basis. The processor then provides a per element saliency measure by combining the first image measure and the second image measure.

Visual saliency estimation for images and video

Systems and methods to generate stereoscopic panoramas obtain images based on captured images. The obtained images may be processed and/or preprocessed, for example to compensate for perspective distortion caused by the non-ideal camera orientation during capturing, to reduce vertical parallax, to align adjacent images, and/or to reduce rotational and/or positional drift between adjacent images. The obtained images may be used for interpolating virtual in-between images on the fly to reduce visible artifacts in the resulting panorama. Obtained and/or interpolated images (or image fragments) may be stitched together to form a stereoscopic panorama.

Stereoscopic panoramas

A device and method incorporates features of a temporal contrast sensor with a camera sensor in an imager. The method includes registering the camera sensor with the temporal contrast sensor as a function of a calibration target. The method includes receiving camera sensor data from the camera sensor and temporal contrast sensor data from the temporal contrast sensor. The method includes generating a plurality of images as a function of incorporating the temporal contrast sensor data with the camera sensor data.

Device and method for calibrating a temporal contrast sensor with a frame-based camera sensor

Methods for content-aware image compression are disclosed. One method comprises the steps of non-uniformly downscaling an original input image according to a saliency map, creating a residual image, encoding the residual image and downscaled input image, and transmitting the residual image and downscaled input image. The encoded image components are transmitted to a receiver. Downscaling may be performed using an aspect ratio that is automatically calculated from the saliency map. The saliency map may be based on an algorithm specified at an encoder or on regions of interest selected by a plurality of users of receivers that receive the transmitted encoded image components.

Content-aware image compression method

Methods and systems for generating stereoscopic content with granular control over binocular disparity based on multi-perspective imaging from representations of light fields are provided. A reference image, a three-dimensional ('3D') representation of a light field corresponding to the reference image, and a goal disparity image that indicates a goal binocular disparity for one or more pixels of the reference image may be received. For each pixel of an output image corresponding to the reference image, a point within the light field that is a closest match for the goal binocular disparity of a corresponding pixel of the goal disparity image may be determined. A stereoscopic image pair including the reference image and the output image may be generated.

Multi-perspective stereoscopy from light fields

A method for performing light-based calibration of optics with caustic surfaces. The method includes mapping a light detecting device to a programmable light source. Then, the method includes operating a calibration light source to direct light onto one or more caustic surfaces of an optical assembly, e.g., an assembly of one or more lenses, facets, lenticules, and lenslets. The method may then involve, with the light detecting device, capturing an image of a projection surface of the optical assembly, which is opposite the one or more caustic surfaces in the optical assembly, as the projection surface is illuminated by the light from the light source. Further, the method includes processing the captured image, along with the mapping of the light detecting device to the programmable light source, to generate a calibration map of the optical assembly including the caustic surfaces.

Light-based caustic surface calibration

Methods and apparatus for visual saliency estimation for images and video are described. In an embodiment, a process includes decomposing, by a processor, an image into elements, wherein each element includes at least one pixel. The processor then calculates a first image measure indicative of each element's uniqueness in the image on a per element basis, and a second image measure indicative of each element's spatial distribution in the image on a per element basis. A per element saliency measure is provided by combining the first image measure and the second image measure, or by utilizing the first image measure, or by utilizing the second image measure.

Growing community of inventors

Zurich, Switzerland

Yael Pritch