Brown University

Max-planck-gesellschaft Zur F&#xf6;rderung Der Wissenschaften E. V.

University of Texas System

Amazon Technologies, Inc.

Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V.

Michael Julian Black

Matthew M Loper

Peng Guan

Alexander W Weiss

Oren Freifeld

Matthew Maverick Loper

Eric Rachlin

Leonid Sigal

Naureen Mahmood

Alexandru O Balan

Timothy S St Clair

Nicolas Heron

Alexander Weiss

Javier Romero Gonzalez-Nicolas

Matthew Q Hill

Stephan Streuber

Carina A Hahn

Gerard Pons-Moll

Javier Romero

Maria Alejandra Quirós Ramírez

David L Hirshberg

Alex Weiss

Silvia Zuffi

Alice O'Toole

David Allen Smith

Steven Douglas Hatch

Sergey Prokudin

Xiaochen Hu

Evan Lee

Jonathan Robert Cilley

William John O'Farrell

Jason Lawrence Gelman

Nikolas Hesse

Theodora Hinkle

Sergi Pujades

Martin Kristiansen

Matthew Loper

Matt Loper

Disclosed are computer-readable devices, systems and methods for generating a model of a clothed body. The method includes generating a model of an unclothed human body, the model capturing a shape or a pose of the unclothed human body, determining two-dimensional contours associated with the model, and computing deformations by aligning a contour of a clothed human body with a contour of the unclothed human body. Based on the two-dimensional contours and the deformations, the method includes generating a first two-dimensional model of the unclothed human body, the first two-dimensional model factoring the deformations of the unclothed human body into one or more of a shape variation component, a viewpoint change, and a pose variation and learning an eigen-clothing model using principal component analysis applied to the deformations, wherein the eigen-clothing model classifies different types of clothing, to yield a second two-dimensional model of a clothed human body.

Parameterized model of 2D articulated human shape

Disclosed are systems and techniques for extracting user body shape (e.g., a representation of the three-dimensional body surface) from user behavioral data. The behavioral data may not be explicitly body-shape-related, and can include shopping history, social media likes, or other recorded behaviors of the user within (or outside of) a networked content delivery environment. The determined body shape can be used, for example, to generate a virtual fitting room user interface.

Machine learning systems and methods for extracting user body shape from behavioral data

Systems and methods are provided for generating an image of a posed human figure or other subject using a neural network that is trained to translate a set of points to realistic images by reconstructing projected surfaces directly in the pixel space or image space. Input to the image generation process may include parameterized control features, such as body shape parameters, pose parameters and/or a virtual camera position. These input parameters may be applied to a three-dimensional model that is used to generate the set of points, such as a sparsely populated image of color and depth information at vertices of the three-dimensional model, before additional image generation occurs directly in the image space. The visual appearance or identity of the synthesized human in successive output images may remain consistent, such that the output is both controllable and predictable.

Image generation from 3D model using neural network

A computer-implemented method for indexing a database of human body shapes, wherein the human body shapes are represented in terms of coefficient vectors in a geometric body space. The method includes predicting, for human body shapes in the database, a vector of word ratings, based on parameters of the human body shape, using a mapping between the geometric body space and a linguistic body space, wherein the linguistic body space is represented in terms of body descriptor words; and storing the body descriptor words and their predicted rating values in the database with each human body shape.

Crowdshaping realisticD avatars with words

A computer-implemented method for automatically obtaining pose and shape parameters of a human body. The method includes obtaining a sequence of digital 3D images of the body, recorded by at least one depth camera; automatically obtaining pose and shape parameters of the body, based on images of the sequence and a statistical body model; and outputting the pose and shape parameters. The body may be an infant body.

Skinned multi-infant linear body model

The invention comprises a learned model of human body shape and pose dependent shape variation that is more accurate than previous models and is compatible with existing graphics pipelines. Our Skinned Multi-Person Linear model (SMPL) is a skinned vertex based model that accurately represents a wide variety of body shapes in natural human poses. The parameters of the model are learned from data including the rest pose template, blend weights, pose-dependent blend shapes, identity-dependent blend shapes, and a regressor from vertices to joint locations. Unlike previous models, the pose-dependent blend shapes are a linear function of the elements of the pose rotation matrices. This simple formulation enables training the entire model from a relatively large number of aligned 3D meshes of different people in different poses. The invention quantitatively evaluates variants of SMPL using linear or dual quaternion blend skinning and show that both are more accurate than a BlendSCAPE model trained on the same data. In a further embodiment, the invention realistically models dynamic soft-tissue deformations. Because it is based on blend skinning, SMPL is compatible with existing rendering engines and we make it available for research purposes.

Skinned multi-person linear model

A method for generating a body shape, comprising: receiving one or more linguistic descriptors related to the body shape; retrieving an association between the one or more linguistic descriptors and a body shape; and generating the body shape, based on the association.

Crowdshaping realistic 3D avatars with words

Present application refers to a method, a model generation unit and a computer program (product) for generating trained models (M) of moving persons, based on physically measured person scan data (S). The approach is based on a common template (T) for the respective person and on the measured person scan data (S) in different shapes and different poses. Scan data are measured with a 3D laser scanner. A generic personal model is used for co-registering a set of person scan data (S) aligning the template (T) to the set of person scans (S) while simultaneously training the generic personal model to become a trained person model (M) by constraining the generic person model to be scan-specific, person-specific and pose-specific and providing the trained model (M), based on the co-registering of the measured object scan data (S).

Co-registration&#x2014;simultaneous alignment and modeling of articulated 3D shapes

Disclosed is a method including receiving an input image including a human, predicting, based on a convolutional neural network that is trained using examples consisting of pairs of sensor data, a corresponding body shape of the human and utilizing the corresponding body shape predicted from the convolutional neural network as input to another convolutional neural network to predict additional body shape metrics.

Machine learning systems and methods of estimating body shape from images

A system and method of estimating the body shape of an individual from input data such as images or range maps. The body may appear in one or more poses captured at different times and a consistent body shape is computed for all poses. The body may appearin minimal tight-fitting clothing or in normal clothing wherein the described method produces an estimate of the body shape under the clothing. Clothed or bare regions of the body are detected via image classification and the fitting method is adapted to treat each region differently. Body shapes are represented parametrically and are matched to other bodies based on shape similarity and other features. Standard measurements are extracted using parametric or non-parametric functions of body shape. The system components support many applications in body scanning, advertising, social networking, collaborative filtering and Internet clothing shopping.

Method and apparatus for estimating body shape

Disclosed is a method including receiving visual input comprising a human within a scene, detecting a pose associated with the human using a trained machine learning model that detects human poses to yield a first output, estimating a shape (and optionally a motion) associated with the human using a trained machine learning model associated that detects shape (and optionally motion) to yield a second output, recognizing the scene associated with the visual input using a trained convolutional neural network which determines information about the human and other objects in the scene to yield a third output, and augmenting reality within the scene by leveraging one or more of the first output, the second output, and the third output to place 2D and/or 3D graphics in the scene.

Machine learning systems and methods for augmenting images

A system generates a clothing deformation model which models one or more of a pose-dependent clothing shape variation which is induced by underlying body pose parameters, a pose-independent clothing shape variation which is induced by clothing size and underlying body shape parameters and a clothing shape variation including a combination of the pose-dependent clothing shape variation and/or the pose-independent clothing shape variation. The system generates, for an input human body, a custom-shaped garment associated with a clothing type by mapping, via the clothing deformation model, body shape parameters of the input human body to clothing shape parameters of the clothing type and dresses the input human body with the custom-shaped garment.

System and method for simulating realistic clothing

A method for providing a three-dimensional body model which may be applied for an animation, based on a moving body, wherein the method comprises providing a parametric three-dimensional body model, which allows shape and pose variations; applying a standard set of body markers; optimizing the set of body markers by generating an additional set of body markers and applying the same for providing 3D coordinate marker signals for capturing shape and pose of the body and dynamics of soft tissue; and automatically providing an animation by processing the 3D coordinate marker signals in order to provide a personalized three-dimensional body model, based on estimated shape and an estimated pose of the body by means of predicted marker locations.

Method for providing a three dimensional body model

The invention comprises a learned model of human body shape and pose dependent shape variation that is more accurate than previous models and is compatible with existing graphics pipelines. Our Skinned Multi-Person Linear model (SMPL) is a skinned vertex based model that accurately represents a wide variety of body shapes in natural human poses. The parameters of the model are learned from data including the rest pose template, blend weights, pose-dependent blend shapes, identity-dependent blend shapes, and a regressor from vertices to joint locations. Unlike previous models, the pose-dependent blend shapes are a linear function of the elements of the pose rotation matrices. This simple formulation enables training the entire model from a relatively large number of aligned 3D meshes of different people in different poses. The invention quantitatively evaluates variants of SMPL using linear or dual-quaternion blend skinning and show that both are more accurate than a Blend SCAPE model trained on the same data. In a further embodiment, the invention realistically models dynamic soft-tissue deformations. Because it is based on blend skinning, SMPL is compatible with existing rendering engines and we make it available for research purposes.

Growing community of inventors

Tuebingen, Germany

Michael Julian Black