Shanghaitech University

Zhiru Shi

Jingyi Yu

Minye Wu

Yi Ma

Yingliang Zhang

Qiang Hu

Wenguang Ma

Ziran Xing

Guli Zhang

A method of processing light field images for separating a transmitted layer from a reflection layer. The method comprises capturing a plurality of views at a plurality of viewpoints with different polarization angles; obtaining an initial disparity estimation for a first view using SIFT-flow, and warping the first view to a reference view; optimizing an objective function comprising a transmitted layer and a secondary layer using an Augmented Lagrange Multiplier (ALM) with Alternating Direction Minimizing (ADM) strategy; updating the disparity estimation for the first view; repeating the steps of optimizing the objective function and updating the disparity estimation until the change in the objective function between two consecutive iterations is below a threshold; and separating the transmitted layer and the secondary layer using the disparity estimation for the first view.

Light field based reflection removal

According to some embodiments, an imaging processing method for extracting a plurality of planar surfaces from a depth map includes computing a depth change indication map (DCI) from a depth map in accordance with a smoothness threshold. The imaging processing method further includes recursively extracting a plurality of planar region from the depth map, wherein the size of each planar region is dynamically adjusted according to the DCI. The imaging processing method further includes clustering the extracted planar regions into a plurality of groups in accordance with a distance function; and growing each group to generate pixel-wise segmentation results and inlier points statistics simultaneously.

System and method for extracting planar surface from depth image

A light field image processing method is disclosed for removing occluding foreground and blurring uninterested objects, by differentiating objects located at different depths of field and objects belonging to distinct categories, to create see-through effects. In various embodiments, the image processing method may blur a background object behind a specified object of interest. The image processing method may also at least partially remove from the rendered image any occluding object that may prevent a viewer from viewing the object of interest. The image processing method may further blur areas of the rendered image that represent an object in the light field other than the object of interest. The method includes steps of constructing a light field weight function comprising a depth component and a semantic component, where the weight function assigns a ray in the light field with a weight; and conducting light field rendering using the weight function.

Light field image rendering method and system for creating see-through effects

A method of detecting and recognizing faces using a light field camera array is provided. The method includes capturing multi-view color images using the light field camera array; obtaining a depth map; conducting light field rendering using a weight function comprising a depth component and a sematic component, where the weight function assigns a ray in the light field with a weight; and detecting and recognizing a face.

Face detection and recognition method using light field camera system

A method of image processing for occlusion removal in images and videos captured by light field camera systems. The method comprises: capturing a plurality of camera views using a plurality of cameras; capturing a plurality of depth maps using a plurality of depth sensors; generating a depth map for each camera view; calculating a target view on a focal plane corresponding to a virtual camera; set a weighting function on the pixels on the camera views based on the depth map and a virtual distance; and blending the pixels in accordance with the weighting function to generate a refocused target view.

Light field system occlusion removal

A method of perceptual video coding based on face detection is provided. The method includes calculating a bit allocation scheme for coding a light field video based on a saliency map of the face, calculating an LCU level Lagrange multiplier for coding a light field video based on a saliency map of the face and calculating an LCU level quantization parameter for coding a light field video based on a saliency map of the face.

Growing community of inventors

Shanghai, China

Zhiru Shi