Bodygram, Inc.

Visualize K.k.

Kyohei Kamiyama

Chong Jin Koh

Yu Sato

Subas Chhatkuli

Nobuyuki Hayashi

Systems and methods for generating a prediction of a body composition of a user using an image capturing device are disclosed. The systems and methods can be used to predict body compositions such as body fat percentage, water content percentage, muscle mass, bone mass, and so on, from a single user image. The methods include the steps of receiving one or more user images and one or more user parameters, generating one or more key points based on the one or more user images, and generating a prediction of the body composition of the user based on the one or more key points and the one or more user parameters, using a body composition deep learning network (DLN). In one embodiment, the body composition DLN comprises a face image DLN, a body feature DLN, and an output DLN.

Predicting body composition from user images using deep learning networks

The present invention provides systems and methods for generating a 3D product mesh model and product dimensions from user images. The system is configured to receive one or more images of a user's body part, extract a body part mesh having a plurality of body part key points, generate a product mesh from an identified subset of the body part mesh, and generate one or more product dimensions in response to the selection of one or more key points from the product mesh. The system may output the product mesh, the product dimensions, or a manufacturing template of the product. In some embodiments, the system uses one or more machine learning modules to generate the body part mesh, identify the subset of the body part mesh, generate the product mesh, select the one or more key points, and/or generate the one or more product dimensions.

Generation of product mesh and product dimensions from user image data using deep learning networks

Disclosed are systems and methods for generating data sets for training deep learning networks for key point annotations and measurements extraction from photos taken using a mobile device camera. The method includes the steps of receiving a 3D scan model of a 3D object or subject captured from a 3D scanner and a 2D photograph of the same 3D object or subject at a virtual workspace. The 3D scan model is rigged with one or more key points. A superimposed image of a pose-adjusted and aligned 3D scan model superimposed over the 2D photograph is captured by a virtual camera in the virtual workspace. Training data for a key point annotation DLN is generated by repeating the steps for a plurality of objects belonging to a plurality of object categories. The key point annotation DLN learns from the training data to produce key point annotations of objects from 2D photographs captured using any mobile device camera.

Methods and systems for generating 3D datasets to train deep learning networks for measurements estimation

A structured 3D model of a real-world object is generated from a series of 2D photographs of the object, using photogrammetry, a keypoint detection deep learning network (DLN), and retopology. In addition, object parameters of the object are received. A pressure map of the object is then generated by a pressure estimation DLN based on the structured 3D model and the object parameters. The pressure estimation DLN was trained on structured 3D models, object parameters, and pressure maps of a plurality of objects belonging to a given object category. The pressure map of the real-world object can be used in downstream processes, such as custom manufacturing.

Methods and systems for predicting pressure maps of 3D objects from 2D photos using deep learning

Display screen with graphical user interface

Disclosed are systems and methods for generating large data sets for training deep learning networks (DLNs) for 3D measurements extraction from 2D images taken using a mobile device camera. The method includes the steps of receiving a 3D model of a 3D object; extracting spatial features from the 3D model; generating a first type of augmentation data for the 3D model, such as but not limited to skin color, face contour, hair style, virtual clothing, and/or lighting conditions; augmenting the 3D model with the first type of augmentation data to generate an augmented 3D model; generating at least one 2D image from the augmented 3D model by performing a projection of the augmented 3D model onto at least one plane; and generating a training data set to train the deep learning network (DLN) for spatial feature extraction by aggregating the spatial features and the at least one 2D image.

Methods and systems for automatic generation of massive training data sets from 3D models for training deep learning networks

Disclosed are systems and methods for body weight prediction from one or more images. The method includes the steps of receiving one or more subject parameters; receiving one or more images containing a subject; identifying one or more annotation key points for one or more body features underneath a clothing of the subject from the one or more images utilizing one or more annotation deep-learning networks; calculating one or more geometric features of the subject based on the one or more annotation key points; and generating a prediction of the body weight of the subject utilizing a weight machine-learning module based on the one or more geometric features of the subject and the one or more subject parameters.

Systems and methods for weight measurement from user photos using deep learning networks

A mobile computing device used for measuring the height of an object, such as a human user, may be positioned on a reference surface, such as the ground plane. The reference surface is detected and a position guide is generated in an augmented reality (AR) plane on a display of the mobile computing device. The AR plane enables the object being measured to be positioned at a measurement position located at a predefined distance along the reference surface from the mobile computing device. The top and bottom of the object are detected in an image taken by the mobile computing device. The height of the object is measured based on the predefined distance and a distance between the top and bottom of the object in the image. The height of the object can also be measured with assistance from software development kits (SDKs) included in the mobile computing device.

Methods and systems for height estimation from a 2D image using augmented reality

Disclosed are systems and methods for generating large data sets for training deep learning networks for 3D measurements extraction from images taken using a mobile device camera. The method includes the steps of receiving at least one 3D model; generating a first type of augmentation data, such as but not limited to skin color, face contour, hair style, virtual clothing, and/or lighting conditions; augmenting the 3D model with the first type of augmentation data; generating at least one image from the augmented 3D model; receiving a second type of augmentation data, such as a plurality of background images representing a variety of backgrounds; augmenting the at least one image with the second type of augmentation data to generate a plurality of augmented images; extracting spatial features from the 3D model; and providing the plurality of augmented images and the spatial features to train a deep learning network for 3D measurement determination.

Disclosed are systems and methods for full body measurements extraction using a mobile device camera. The method includes the steps of receiving one or more user parameters from a user device; receiving at least one image from the user device, the at least one image containing the human and a background; performing body segmentation on the at least one image to identify one or more body features associated with the human from the background; performing annotation on the one or more identified body features to generate annotation lines on each body feature corresponding to body feature measurement locations utilizing a plurality of annotation deep-learning networks that have been separately trained on each body feature; generating body feature measurements from the one or more annotated body features utilizing a sizing machine-learning module based on the annotated body features and the one or more user parameters; and generating body size measurements by aggregating the body feature measurements for each body feature.

Growing community of inventors

Tokyo, Japan

Kyohei Kamiyama