Facebook Technologies, LLC

Gabriel Bailowitz Schwartz

Jason Saragih

Tomas Simon Kreuz

Stephen Anthony Lombardi

Shih-En Wei

Shugao Ma

In one embodiment, a system may access an input image of an object captured by cameras, and the input image depicts appearance information associated with an object. The system may generate a first mesh of the object based on features identified from the input image of the object. The system may generate, by processing the first mesh using a machine-learning model, a position map that defines a contour of the object. Each pixel in the position map corresponds to a three-dimensional coordinate. The system may further generate a second mesh based on the position map, wherein the second mesh has a higher resolution than the first mesh. The system may render an output image of the object based on the second mesh. The system disclosed in the present application can render a dense mesh which has a higher resolution to provide details which cannot be compensated by texture information.

Systems and methods for rendering avatar with high resolution geometry

In one embodiment, a method includes accessing a number of pictures of an object, constructing a modeling volume for three-dimensional modeling of the object by processing the number of pictures using a machine-learning framework, where the modeling volume is associated with a number of color and opacity information that are associated with a number of regions in the modeling volume, and rendering an image of the object from a view-point using the modeling volume, where each pixel of the image is rendered by projecting a virtual ray from the view-point and through the modeling volume, determining one or more of the number of regions in the modeling volume intersected by the virtual ray, and determining a color and an opacity of the pixel based on an accumulation of the color and opacity information associated with the one or more of the number of regions intersected by the virtual ray.

Three-dimensional modeling volume for rendering images

In one embodiment, a system may capture one or more images of a user using one or more cameras, the one or more images depicting at least an eye and a face of the user. The system may determine a direction of a gaze of the user based on the eye depicted in the one or more images. The system may generate a facial mesh based on depth measurements of one or more features of the face depicted in the one or more images. The system may generate an eyeball texture for an eyeball mesh by processing the direction of the gaze and the facial mesh using a machine-learning model. The system may render an avatar of the user based on the eyeball mesh, the eyeball texture, the facial mesh, and a facial texture.

Growing community of inventors

Pittsburgh, PA, United States of America

Gabriel Bailowitz Schwartz