Hangzhou City University

Junhao Huang

Anping Wan

Xiaomin Cheng

Kaiyang Wang

Junjie Jiang

A generalizable neural radiation field reconstruction method based on multi-modal information fusion, including: Step 1, constructing photometric features and geometric features based on unstructured multi-views, and constructing a multi-modal neural encoder by performing incrementally complementary fusion on the photometric features and the geometric features; Step 2, converting the multi-modal neural encoder and raw RGB pixel bodies of the unstructured multi-views into a volume density and radiation brightness; Step 3, sampling light on the basis of the constructed multi-modal neural encoder, aggregating context features of the sampled light based on a transformer network to obtain light context features; and Step 4, decoding, using the light context features, the volume density and the radiation brightness; rendering, based on the decoded volume density and the radiation brightness, to generate a free-view RGB-D image; and guiding dense reconstruction of a low-texture scene by combining photometric supervision and sparse geometric supervision.

Growing community of inventors

Zhejiang, China

Junhao Huang