Computer system and method for reconciling data

Abstract

A system and method are provided for reconciling data used by a data management system. In some examples, the data management system provides a workspace for machine learning. The method includes obtaining an input dataset, the input dataset being replicated from a baseline dataset to enable the data management system to operate on the input dataset. The method also includes comparing the input dataset to the baseline data set to determine discrepancies between the input and baseline datasets, by, for each of a plurality of database pools, process data assigned to that pool by concurrently checking for the discrepancies and executing statements without waiting for all pools to have finished processing. The method also includes creating a delta table for each pool to identify extracted data associated with the discrepancies; and combining delta pools from the plurality of database pools and process columns in the delta table.