International Business Machines Corporation

Md Arafat Sultan

Vittorio R Castelli

Robert G Farrell

Michael Robert Glass

Md Faisal Mahbub Chowdhury

Nicholas Brady Garvan Monath

Ramon Astudillo

Shubham Chandel

Embodiments relate to an artificial intelligence (AI) computer platform to incorporate synthetic data and ground truth data, and to promote diversity and accuracy in generating the synthetic data. Synthetic questions are generated by a question generator in response to semantically related ground truth passage and answer data. Each generated question is presented to an answer generator together with the semantically related ground truth passage. Each synthetic question is evaluated with respect to its diversity from previous synthetic questions generated for the same ground truth passage and answer data. Each synthetic question is also evaluated with respect to the accuracy of the answer generated by the answer generator. A reward function that captures both accuracy and diversity of each synthetic question is leveraged to selectively modify the question generator, with the selective modification(s) directed at increasing textual diversity and maintaining accuracy of the generated synthetic questions.

Automatically generating diverse text

Training a machine learning model such as a neural network, which can automatically extract a hypernym from unstructured data, is disclosed. A preliminary candidate list of hyponym-hypernym pairs can be parsed from the corpus. A preliminary super-term&#x2014;sub-term glossary can be generated from the corpus, the preliminary super-term&#x2014;sub-term glossary containing one or more super-term&#x2014;sub-term pairs. A super-term&#x2014;sub-term pair can be filtered from the preliminary super-term&#x2014;sub-term glossary, responsive to detecting that the super-term&#x2014;sub-term pair is not a candidate for hyponym-hypernym pair, to generate a final super-term&#x2014;sub-term glossary. The preliminary candidate list of hyponym-hypernym pairs and the final super-term&#x2014;sub-term glossary can be combined to generate a final list of hyponym-hypernym pairs. An artificial neural network can be trained using the final list of hyponym-hypernym pairs as a training data set, the artificial neural network trained to identify a hypernym given new text data.

Unsupervised hypernym induction machine learning

Growing community of inventors

Croton-on-Hudson, NY, United States of America

Md Arafat Sultan