Style-controlled generation of visual assets for video games using machine learning

Abstract

This specification describes a computing system for generating visual assets for video games. The computing system comprises an image segmentation model, a first 3D generation model, and a second 3D generation model. At least one of the first 3D generation model and the second 3D generation model comprises a machine-learning model. The system is configured to obtain: (i) a plurality of images corresponding to the visual asset, each image showing a different view of an object to be generated in the visual asset, and (ii) orientation data for each image that specifies an orientation of the object in the image. A segmented image is generated for each image. This comprises processing the image using the image segmentation model to segment distinct portions of the image into one or more classes of a predefined set of classes. For each image, 3D shape data is generated for a portion of the object displayed in the image. This comprises processing the segmented image of the image, the orientation data of the image, and style data for the visual asset using the first 3D generation model. 3D shape data is generated for the visual asset. This comprises processing the generated 3D shape data of each image using the second 3D generation model.