International Business Machines Corporation

Yang Zhao

Bishwaranjan Bhattacharjee

Issei Yoshida

Michele Merler

Masayasu Muraoka

Graeme William Blackwood

Systems, computer-implemented methods, and computer program products to facilitate improving a mono-lingual language model using pre-trained models and parallel data are provided. According to an embodiment, a system can comprise a processor that executes components stored in memory. The computer executable components can comprise a training component that can use a first language model to improve a mono-lingual second language model through mask language modeling (MLM) training and contrastive learning. The computer executable components can comprise a data component that can collect parallel data for the first language model and the mono-lingual second language model. The computer executable components can comprise a tokenizing component that can tokenize the parallel data for the first language model and the mono-lingual second language model. The computer executable components can comprise an alignment component that can create an alignment of the tokenized parallel data.

Mono-lingual language models using parallel data

A computer-implemented method, a computer program product, and a computer system for dependency tree-based data augmentation for sentence well-formedness judgement. A computer applies a dependency parser to generate a dependency tree for a sentence. The computer removes one or more nodes in the dependency tree, according to a removal ratio for a predetermined rating score. The computer generates, from the dependency tree, a partial tree for the sentence. The computer outputs a rated sentence based on the partial tree. The rated sentence is used as training data.

Growing community of inventors

Tokyo, Japan

Yang Zhao