Identifying poisoned data during data curation using false positive detection

Abstract

Methods and systems for curating data by a data manager are disclosed. Data may be curated from various data sources before being supplied to downstream consumers, and the downstream consumers may rely on the trustworthiness of the curated data to facilitate desired computer-implemented services. Therefore, to identify data (e.g., poisoned data) that, if provided to downstream consumers may negatively impact the desired computer-implemented services, the data may undergo anomaly detection. Data anomalies indicating the presence of poisoned data may be identified based on anomaly patterns exhibited by the data. However, an anomaly pattern may be likely to appear by chance (e.g., may incorrectly indicate the presence of poisoned data) in large volumes of complex data. Therefore, anomaly detection may be coupled with false positive detection in order to manage false positive anomalies. When a true positive anomaly (e.g., poisoned data) is detected, the poisoned portion may be identified and/or remediated.