Method and a system for normalizing data in data processing pipelines

Abstract

The present disclosure provides a method and system for normalizing data related to assets in a business environment. The method comprises receiving at least one data from one or more assets within an enterprise and parsing the at least one data to identify at least one asset type. The method further comprises dynamically generating a structured query language (SQL) query corresponding to the at least one asset type and storing the SQL query in a metadata repository. The method further comprises executing the SQL query from the metadata repository on the at least one data. The method further comprises normalizing the at least one data by merging the results of the executed SQL query with the at least one data, and at least one configuration data to populate a dimensional model in real time, wherein the at least one configuration data comprises metadata associated with the one or more assets and an ontology model representing at least one of attributes, characteristics and relationships associated with the one or more assets. The method further comprises dynamically updating the at least one configuration data with the normalized data. A data processing system for normalizing data related to assets is also disclosed.