Controlled text generation with supervised representation disentanglement and mutual information minimization

Abstract

A computer-implemented method is provided for disentangled data generation. The method includes accessing, by a bidirectional Long Short-Term Memory (LSTM) with a multi-head attention mechanism, a dataset including a plurality of pairs each formed from a given one of a plurality of input text structures and given one of a plurality of style labels for the plurality of input text structures. The method further includes training the bidirectional LSTM as an encoder to disentangle a sequential text input into disentangled representations comprising a content embedding and a style embedding based on a subset of the dataset. The method also includes training a unidirectional LSTM as a decoder to generate a next text structure prediction for the sequential text input based on previously generated text structure information and a current word, from a disentangled representation with the content embedding and the style embedding.