Method and system for characterizing unknown annotator and its type system with respect to reference annotation types and associated reference taxonomy nodes

Abstract

An unknown annotator and its annotation type system are compared against a reference annotation type system. The comparison is done by providing a plurality of documents, and annotating each document using the reference set of document annotators, producing instances of reference annotation types, to generate a pre-annotated reference document set, and using the subject annotator and its subject annotation type system to generate a pre-annotated evaluation document set. Documents in the pre-annotated evaluation document set are compared to documents in the pre-annotated reference document set, and matches in location, within the compared documents, of instances of the subject annotation types and the reference annotation types are identified. Based on the matching data, reference document annotation types are selected that sufficiently correlate with the subject annotation type system. Based on the type system matching, the subject annotator and its type system are associated with a plurality of reference industrial taxonomy nodes.