Naver Corporation

Alexandre Bérard

Laurent Besacier

Ioan Calapodescu

Ahmet Üstün

Matthias Gallé

Fahimeh Saleh

Methods and systems for unsupervised training for a neural multilingual sequence-to-sequence (seq2seq) model. Denoising adapters for each of one or more languages is inserted into an encoder and/or a decoder of the seq2seq model. Parameters of the one or more denoising adapters are trained on a language-specific denoising task using monolingual text for each of the one or more languages. Cross-attention weights of the seq2seq model with the trained denoising adapter layers are fine-tuned on a translation task in at least one of the one or more languages with parallel data.

Multilingual unsupervised neural machine translation with denoising adapters

In methods for training a natural language generation (NLG) model using a processor a document-level machine translation (MT) model is provided by training an MT model to receive as input, token sequences in a first language, and to generate as output, token sequences in a second language. An augmented document-level MT model is provided by training the document-level MT model to receive as input, paired language-independent structured data and token sequences in the first language, and to generate as output, token sequences in the second language. The augmented document-level MT model is trained to receive as input, language-independent structured data, and to generate as output, token sequences in the second language.

Growing community of inventors

Grenoble, France

Alexandre Bérard