Technical data enrichment through language models

Abstract

This disclosure provides a mechanism for the enrichment of sparse datasets using language models. By training language models on the specific distribution of known values in a dataset, missing values can be predicted, and the predicted values added, thereby resulting in a more complete dataset. This method also facilitates the enhancement and augmentation of datasets by predicting values for new properties that were not previously available. The approach proves particularly effective at scale, transforming large sparse datasets into more complete and enhanced datasets. Masking language modeling may be employed to train language models capable of generating representations of technical data. Training data includes corpuses of technical data that may be represented as text strings. These pretrained models are fine-tuned to predict various properties. The resulting models can predict missing values in large technical datasets, providing valuable data for guiding scientific research.