Capital One Services, LLC

Jon Stofer

Jose Moreno

Thomas J O'Connor

William Ye

Samuel Joshua Bennett

Systems and methods of the present disclosure enable anomaly detection based on data lineage by using at least one processor to receive initial lineage records that record initial changes in data items at each data processor of a lineage stream. The processor(s) generates a historical distribution as a data processor baseline for a particular data processor based on each change in each data item at the particular data processor. For lineage records in a subsequent time period, the processor(s) identifies subsequent changes in data items at the particular data processor and generates a real-time dynamic distribution to represent a current data processor behavior for the particular data processor based on each change for the subsequent lineage records. The processor(s) determines a deviation between the data processor baseline and the current data processor behavior based on the historical distribution and the real-time dynamic distribution to identify a data processor anomaly.

Systems and methods for data lineage-based system anomaly detection

In certain embodiments, machine learning and lineage data may be used to manage data. In some embodiments, a computing system may use lineage data to identify two datasets that may be related. The computing system may determine that a user has access to a derivative dataset but does not have access to an original dataset that was used to create the derivative dataset. In response, the computing system may use a machine learning model to generate a similarity score indicating a level of similarity between the original dataset and the derivative dataset. If the similarity score satisfies a threshold score, the computing system may modify access rights of the user so that the user is unable to access a portion of the data in the derivative dataset.

Growing community of inventors

McLean, VA, United States of America

Jon Stofer