International Business Machines Corporation

Masayasu Muraoka

Tetsuya Nasukawa

Issei Yoshida

Khan Md Anwarus Salam

Yang Zhao

Bishwaranjan Bhattacharjee

Yulong Li

Michele Merler

Graeme William Blackwood

Systems, computer-implemented methods, and computer program products to facilitate improving a mono-lingual language model using pre-trained models and parallel data are provided. According to an embodiment, a system can comprise a processor that executes components stored in memory. The computer executable components can comprise a training component that can use a first language model to improve a mono-lingual second language model through mask language modeling (MLM) training and contrastive learning. The computer executable components can comprise a data component that can collect parallel data for the first language model and the mono-lingual second language model. The computer executable components can comprise a tokenizing component that can tokenize the parallel data for the first language model and the mono-lingual second language model. The computer executable components can comprise an alignment component that can create an alignment of the tokenized parallel data.

Mono-lingual language models using parallel data

A computer-implemented method, a computer program product, and a computer system for dependency tree-based data augmentation for sentence well-formedness judgement. A computer applies a dependency parser to generate a dependency tree for a sentence. The computer removes one or more nodes in the dependency tree, according to a removal ratio for a predetermined rating score. The computer generates, from the dependency tree, a partial tree for the sentence. The computer outputs a rated sentence based on the partial tree. The rated sentence is used as training data.

Dependency tree-based data augmentation for sentence well-formedness judgement

A computer-implemented method is provided for data augmentation. The method includes receiving a set of different base models already pretrained and a set of different test cases. The method further includes collecting a plurality of prediction results of the set of different test cases from the set of different base models. The method also includes identifying a test case as a candidate for the data augmentation based on a number of models in the set of different base models which fail to solve the test case. The method additionally includes augmenting, by a processor device, the identified test case with additional data to form an augmented training dataset. The method further includes retraining at least some of the different base models with the augmented training dataset.

Data augmentation based on failure cases

A computer-implemented method is provided for data augmentation. The method includes calculating, by a hardware processor for each of words in a text data, a word replacement probability based on a word occurrence frequency in the text data, wherein the word replacement probability decreases with increasing word occurrence frequency. The method additionally includes selectively replacing at least one of the words in the text data with words predicted therefor by a Bidirectional Neural Network Language Model (BiNNLM) to generate augmented text data, based on the word replacement probability.

Data augmentation by dynamic word replacement

A computer-implemented method for identifying an expression for a target concept, includes: obtaining a set of texts as a target set of texts, with each text being associated with one of images relevant to a target concept. Candidate expressions for the target concept are extracted from the target set of texts. The candidate expressions are characteristic of the target set of texts. Each image relevant to one of the candidate expressions is collected by using an image search engine. A target expression for the target concept is selected from the candidate expressions based on a comparison result of the target concept and the collected images.

Identifying expressions for target concept with images

A computer-implemented method for finding an asymmetric relation between a plurality of target words is disclosed. The method includes preparing a plurality of image sets, each of which includes one or more images relevant to a corresponding one of the plurality of the target words. The method also includes obtaining a plurality of object labels for each of the plurality of image sets. The method further includes computing a representation for each of the plurality of the target words using the plurality of the object labels obtained for each of the plurality of image sets. The method includes further determining whether there is an asymmetric relation between the plurality of the target words using representations computed for the plurality of the target words.

Growing community of inventors

Tokyo, Japan

Masayasu Muraoka