Personalized avatars using text description of features

Abstract

Systems and methods for generating personalized avatars using text description of features are provided. An example method includes receiving a first image that includes a face, acquiring a text including a first prompt for stylization of the first image, generating a second prompt concerning the face, combining the first prompt and the second prompt into a third prompt, encoding the third prompt into a text embedding, encoding the first image into an image embedding, randomly generating a first noise for the image embedding, adding the first noise to the image embedding to obtain a noisy image embedding, providing the noisy image embedding and the text embedding to a first neural network to generate a second noise, modifying the noisy image embedding based on the second noise, and decoding, using a second neural network, the noisy image embedding into a second image including the face stylized according to the third prompt.