Capital One Services, LLC

Samuel Joshua Bennett

Jose Moreno

Saianirudh Kantabathina

Alec Sokol

Selwyn Lehmann

Christopher Cellucci

Systems and methods of the present disclosure enable anomaly detection based on data lineage by using at least one processor to receive initial lineage records that record initial changes in data items at each data processor of a lineage stream. The processor(s) generates a historical distribution as a data processor baseline for a particular data processor based on each change in each data item at the particular data processor. For lineage records in a subsequent time period, the processor(s) identifies subsequent changes in data items at the particular data processor and generates a real-time dynamic distribution to represent a current data processor behavior for the particular data processor based on each change for the subsequent lineage records. The processor(s) determines a deviation between the data processor baseline and the current data processor behavior based on the historical distribution and the real-time dynamic distribution to identify a data processor anomaly.

Systems and methods for data lineage-based system anomaly detection

In some implementations, a system may receive information identifying a dataset. The system may process an identification attribute using a function that generates a first value, to generate a first identifier for the dataset. The system may search a data store storing a plurality of groupings to identify a grouping with the first identifier for the dataset. The system may extract a second identifier from the grouping with the first identifier for the dataset. The system may search a data lineage based graph representation of a plurality of datasets to identify a graph node representing the dataset. The system may update the data lineage based graph representation of the plurality of datasets to link the dataset with the at least one other dataset based on searching the data lineage based graph representation of the plurality of datasets to identify the graph node.

Growing community of inventors

Midlothian, VA, United States of America

Samuel Joshua Bennett