Spree3d Corporation

Amr A Ali

Ahmed A Ewais

Mohamed N Moustafa

Apparati, methods, and computer readable media for inserting identity information from a source image (static image or video) () into a destination video (), while mimicking motion of the destination video (). In an apparatus embodiment, an identity encoder () is configured to encode identity information of the source image (). When source image () is a multi-frame static image or a video, an identity code aggregator () is positioned at an output of the identity encoder (), and produces an identity vector (). A driver encoder () is coupled to the destination (driver) video (), and has two components: a pose encoder () configured to encode pose information of the destination video (), and a motion encoder () configured to separately encode motion information of the destination video (). The driver encoder () produces two vectors: a pose vector () and a motion vector (). A neural network generator () has three inputs: the identity vector (), the pose vector (), and the motion vector (). The neural network generator () is configured to generate, in response to these three inputs, a composite video () comprising identity information of the source image () inserted into the destination video (), where the composite video () has substantially the same temporal information as the destination video ().

Video reenactment taking into account temporal information

Apparati, methods, and computer readable media for inserting identity information from a source image () into a destination image (), while mimicking illumination of the destination image (). In an apparatus embodiment, an identity encoder () is configured to encode just identity information of the source image () and to produce an identity vector (), where the identity encoder () does not encode any pose information or illumination information of the source image (). A driver encoder () has two components: a pose encoder () configured to encode pose information of the destination image () and an illumination encoder () configured to separately encode illumination information of the destination image (), and to produce two vectors: a pose vector () and an illumination vector (). A neural network generator () is coupled to the identity encoder () and to the driver encoder (), and has three inputs: the identity vector (), the pose vector (), and the illumination vector (). The neural network generator () is configured to generate, in response to these three inputs, a composite image () comprising identity information of the source image () inserted into the destination image (), wherein the composite image has substantially the same illumination as the destination image.

Image reenactment with illumination disentanglement

Methods and apparati for inserting face and hair information from a source video () into a destination (driver) video () while mimicking pose, illumination, and hair motion of the destination video (). An apparatus embodiment comprises an identity encoder () configured to encode face and hair information of the source video () and to produce as an output an identity vector; a pose encoder () configured to encode pose information of the destination video () and to produce as an output a pose vector; an illumination encoder () configured to encode head and hair illumination of the destination video () and to produce as an output an illumination vector; and a hair motion encoder () configured to encode hair motion of the destination video () and to produce as an output a hair motion vector. The identity vector, pose vector, illumination vector, and hair motion vector are fed as inputs to a neural network generator (). The neural network generator () is configured to generate, in response to the four inputs, a composite video () comprising face and hair information from the source video () inserted into the destination video ().

Growing community of inventors

Cairo, Egypt

Amr A Ali