Massive scale heterogeneous data ingestion and user resolution

Abstract

This disclosure relates to data association, attribution, annotation, and interpretation systems and related methods of efficiently organizing heterogeneous data at a massive scale. Incoming data is received and extracted for identifying information ('information'). Multiple dimensionality reducing functions are applied to the information, and based on the function results, the information are grouped into sets of similar information. Filtering rules are applied to the sets to exclude non-matching information in the sets. The sets are then merged into groups of information based on whether the sets contain at least one common information. A common link may be associated with information in a group. If the incoming data includes the identifying information associated with to the common link, the incoming data is assigned the common link. In some embodiments, incoming data are not altered but assigned into domains.