Sony Group Corporation

Sony Corporation of America

US Government as Represented by the Secretary of the Army

Chen Fu

Mohammad Gharavi-Alkhansari

Mohammad Gharavi Alkhansari

Gonzalo R Arce

Michael L Don

An electronic apparatus and method for generation of subsurface scattering texture maps for three-dimensional (3D) objects is provided. The electronic apparatus controls a set of light sources to generate a set of lighting patterns. The electronic apparatus controls a set of image capture devices to capture a set of images of an object that is illuminated by the generated set of lighting patterns. The electronic apparatus determines pixel-level offset information based on application of an image registration operation on the set of images. The electronic apparatus generates a set of corrected images based on the pixel-level offset information and the set of images. The electronic apparatus estimates subsurface scattering (SSS) parameters in a form of texture maps corresponding to a 3D mesh of the object. The estimation is performed based on a fitting of a scattering model on the set of corrected images.

Generation of subsurface scattering texture maps for 3D objects

An electronic device and method for lighting control for multi-view image capture is provided. The electronic device selects, from a plurality of predefined lighting patterns, a set of lighting patterns to capture a set of images of a subject. The electronic device controls a lighting system to illuminate the subject with a first pulse and a second pulse. An illumination intensity increases from a first intensity to a second intensity, associated with the first pulse and the second pulse respectively. The electronic device controls the lighting system to illuminate the subject with flash pulses corresponding to the selected set of lighting patterns, based on illumination of the subject with the first pulse and the second pulse. The electronic device controls a set of multi-view image capture devices to capture the set of images of the subject, based on illumination of the subject with the selected set of flash pulses.

Lighting control for multi-view image capture

An electronic device and method for generation of reflectance maps for relightable 3D models is disclosed. The electronic device acquires multi-view image data that includes a set of images of an object and generates a 3D mesh of the object based on the multi-view image data. The electronic device obtains a set of motion-corrected images based on a minimization of a rigid motion associated with the object between images of the set of images and generates texture maps in a UV space based on the set of motion-corrected images and the 3D mesh. The electronic device obtains specular and diffuse reflectance maps based on a separation of specular and diffuse reflectance components from the texture maps, and obtains a relightable 3D model of the object based on the specular and diffuse reflectance maps.

Generation of reflectance maps for relightable 3D models

An electronic device and method for 3D human modeling under a specific body-fitting of clothes is disclosed. A set of depth images of a person wearing clothes is acquired and an initial body model is generated based on a first shape-fitting of a human body prior with at least a first depth scan. A posed-body model is obtained based on a pose-fitting of the initial body model with at least one depth scan. In each depth scan, clothed scan points and unclothed scan points are determined. A final body model is determined based on a second shape-fitting of vertices which belong to the posed-body model and correspond to an under-cloth skin portion of the body of the person. The second shape-fitting is based on the clothed scan points and the unclothed scan points. The final body model is textured based on a skin texture map for the body.

Three-dimensional (3D) human modeling under specific body-fitting of clothes

A system and method for three-dimensional (3D) microgeometry and reflectance modeling is provided. The system receives images comprising a first set of images of a face and a second set of images of the face. The faces in the first set of images and the second set of images are exposed to omni-directional lighting and directional lighting, respectively. The system generates a 3D face mesh based on the received images and executes a set of skin-reflectance modeling operations by using the generated 3D face mesh and the second set of images, to estimate a set of texture maps for the face. Based on the estimated set of texture maps, the system texturizes the generated 3D face mesh. The texturization includes an operation in which texture information, including microgeometry skin details and skin reflectance details, of the estimated set of texture maps is mapped onto the generated 3D face mesh.

3D microgeometry and reflectance modeling

An electronic device and method for human skin detection based on a human body-prior is provided. A color image of a person is acquired, and a 3D body model of the person is estimated based on the color image. One or more unclothed parts of the 3D body model are identified. The one or more unclothed parts correspond to one or more body parts, of which at least a portion of skin remains uncovered by clothes. From the color image, pixel information corresponding to the one or more unclothed parts is extracted and classification information is determined based on the pixel information. The classification information includes a set of values, each indicating a likelihood of whether a corresponding pixel of the color image is part of an unclothed skin portion of the person. The unclothed skin portion is detected in the color image based on the classification information.

Human skin detection based on human-body prior

Systems, methods, architectures, mechanisms and/or apparatus configured to provide a compressive spectral imager for estimating a spectral image of a rotating scene such as a target scene as viewed from a spinning munition.

Growing community of inventors

San Jose, CA, United States of America

Chen Fu