International Business Machines Corporation

Alon Halfon

Noam Slonim

Ranit Aharonov

Yosi Mass

Charles Arthur Jochim

Ilya Shnayderman

Roy N Bar-Haim

Orith Toledo-Ronen

Eyal Shnarch

Lena Dankin

Liat Ein Dor

Amir Menczel

Ariel Gera

Elad Venezian

Leshem Choshen

An example system includes a processor to pre-train a transformer-based language model on a general domain. The processor can inter-train the pre-trained transformer-based language model using partitioning and classification to generate an inter-trained transformer-based pre-trained language model. The processor can then fine-tune the inter-trained transformer-based pre-trained language model on a target task to generate a fine-tuned transformer-based language model.

Inter-training of pre-trained transformer-based language models using partitioning and classification

A method including: generating, from a text corpus, a lexicon of unigrams and bigrams comprising an embedding for each of said unigrams and bigrams; training a machine learning classifier on a training set comprising a subset of said lexicon, wherein each of said unigrams and bigrams in said subset has a sentiment label; applying said machine learning classifier to said lexicon, to (i) predict a sentiment of each of said unigrams and bigrams, and (ii) update said lexicon with the predicted sentiments; and performing statistical analysis on said updated lexicon, to extract one or more sentiment composition lexicons, wherein each of said one or more sentiment composition lexicons is associated with a sentiment composition class.

Learning sentiment composition from sentiment lexicons

A method of estimating a thematic similarity of sentences, comprising receiving a corpus of a plurality of documents describing a plurality of topics where each document comprises a plurality of sentences arranged in a plurality of sections, constructing sentence triplets for at least some of the sentences, each sentence triplet comprising a respective sentence, a respective positive sentence selected randomly from the section comprising the respective sentence and a respective negative sentence selected randomly from another section, training a first neural network with the sentence triplets to identify sentence-sentence vectors mapping each sentence with a shorter distance to its respective positive sentence compared to the distance to its respective negative sentence and outputting the first neural network for estimating thematic similarity between a pair of sentences by computing a distance between the sentence-sentence vectors produced for each sentence of the pair by the first neural network.

Learning thematic similarity metric from article text units

Growing community of inventors

Rishon Lezion, Israel

Alon Halfon