Document tagging and retrieval using entity specifiers

Abstract

Techniques for managing big data include tagging of documents and subsequent retrieval using per-subject dictionaries having entries with some entries specially designated as entities. An entity indicates that the term in the entry has special meaning, e.g., brands (trademarks/service marks), trade names, geographic identifiers or other classes of terms. A dictionary may include a non-entity entry for a term and one or more entity entries, for different entity types. The entries may also include subject-determining-power scores. The subject-determining-power scores provide an indication of the descriptive power of the term with respect to the subject of the dictionary containing the term. The same term may have entries in multiple dictionaries with different subject-determining-power scores in each of the dictionaries. The entity distinctions for a term can then be used in tagging documents and processing retrieval requests.