International Business Machines Corporation

Neil E Bartlett

Craig A Statchuk

A mechanism is provide for preparing a high-quality data repository. Data and related metadata from a set of data sources are ingested thereby forming a set of unprepared data. The set of unprepared data is transformed based on a set of functions into a set of transformed data. A set of semantic text descriptions that detail the transformation of the set of unprepared data to the set of transformed data is generated using a first set of semantic associations, a second set of semantic associations, and a set of semantic transformation associations. The set of transformed data is tested against one or more governance policies that tracks data lineage to ultimately show that prepared data is in compliance. Responsive to the set of transformed data adhering to the one or more governance policies, a high-quality data repository is automatically built using the transformed data.

Preparing high-quality data repositories sets utilizing heuristic data analysis

A mechanism is provided for preparing a high-quality data repository. Data and related metadata from a set of data sources are ingested thereby forming a set of unprepared data. The set of unprepared data is transformed based on a set of functions into a set of transformed data. A set of semantic text descriptions that detail the transformation of the set of unprepared data to the set of transformed data is generated using a first set of semantic associations, a second set of semantic associations, and a set of semantic transformation associations. The set of transformed data is tested against one or more governance policies that tracks data lineage to ultimately show that prepared data is in compliance. Responsive to the set of transformed data adhering to the one or more governance policies, a high-quality data repository is automatically built using the transformed data.

Growing community of inventors

Toronto, Canada

Neil E Bartlett