Shanghaitech University

Plex-vr Digital Technology (shanghai) Co., Ltd.

Minye Wu

Jingyi Yu

Zhiru Shi

Lan Xu

Yingliang Zhang

Haimin Luo

Wentao Lyu

Wenguang Ma

Anpei Chen

A deep neural network based hair rendering system is presented to model high frequency component of furry objects. Compared with existing approaches, the present method can generate photo-realistic rendering results. An acceleration method is applied in our framework, which can speed up training and rendering processes. In addition, a patch-based training scheme is introduced, which significantly increases the quality of outputs and preserves high frequency details.

Hair rendering system based on deep neural network

A method of processing light field images for separating a transmitted layer from a reflection layer. The method comprises capturing a plurality of views at a plurality of viewpoints with different polarization angles; obtaining an initial disparity estimation for a first view using SIFT-flow, and warping the first view to a reference view; optimizing an objective function comprising a transmitted layer and a secondary layer using an Augmented Lagrange Multiplier (ALM) with Alternating Direction Minimizing (ADM) strategy; updating the disparity estimation for the first view; repeating the steps of optimizing the objective function and updating the disparity estimation until the change in the objective function between two consecutive iterations is below a threshold; and separating the transmitted layer and the secondary layer using the disparity estimation for the first view.

Light field based reflection removal

An image-based method of modeling and rendering a three-dimensional model of an object is provided. The method comprises: obtaining a three-dimensional point cloud at each frame of a synchronized, multi-view video of an object, wherein the video comprises a plurality of frames; extracting a feature descriptor for each point in the point cloud for the plurality of frames without storing the feature descriptor for each frame; producing a two-dimensional feature map for a target camera; and using an anti-aliased convolutional neural network to decode the feature map into an image and a foreground mask.

Multi-view neural human rendering

A method of detecting and recognizing faces using a light field camera array is provided. The method includes capturing multi-view color images using the light field camera array; obtaining a depth map; conducting light field rendering using a weight function comprising a depth component and a sematic component, where the weight function assigns a ray in the light field with a weight; and detecting and recognizing a face.

Face detection and recognition method using light field camera system

A method of image processing for occlusion removal in images and videos captured by light field camera systems. The method comprises: capturing a plurality of camera views using a plurality of cameras; capturing a plurality of depth maps using a plurality of depth sensors; generating a depth map for each camera view; calculating a target view on a focal plane corresponding to a virtual camera; set a weighting function on the pixels on the camera views based on the depth map and a virtual distance; and blending the pixels in accordance with the weighting function to generate a refocused target view.

Light field system occlusion removal

The present invention discloses a method for real-time rendering of giga-pixel images. Image data are subject to offline pre-processing, and then are subject to data decoding and redirection through a decoding module. A corresponding scheduling strategy is determined according to different image inputs, and rendering is executed by a renderer. The present invention may realize the real-time rendering of a giga-pixel panoramic view in a conventional display device, to greatly reduce the resource allocated for rendering of giga-pixel images. The present invention may render an image originally requiring a 40G or more video memory capacity on a common video card with a 1G-4G video memory capacity.

Growing community of inventors

Shanghai, China

Minye Wu