Graph-based labeling rule augmentation for weakly supervised training of machine-learning-based named entity recognition

Abstract

Systems and methods for training a machine-learning model for named-entity recognition. A rule graph is constructed including a plurality of nodes each corresponding to a different labeling rule of a set of labeling rules (including a set of seeding rules of known labeling accuracy and a plurality of candidate rules of unknown labeling accuracy). The nodes are coupled to other nodes based on which rules exhibit the highest sematic similarity. A labeling accuracy metric is estimated for each candidate rule by propagating a labeling confidence metric through the rule graph from the seeding rules to each candidate rule. A subset of labeling rules is then identified by ranking the rules by their labeling confidence metric. The identified subset of labeling rules is applied to unlabeled data to generate a set of weakly labeled named entities and the machine-learning model is trained based on the set of weakly labeled named entities.