International Business Machines Corporation

Peter Urbanetz

John G Rooney

Birgit Monika Pfitzmann

William Robert Reohr

Daniel Nikolaus Bauer

Luis Garcés Erice

Amitkumar Manoharrao Paradkar

Nikolai A Joukov

Laura A Richardson

Kevin M Kingsbury

William B Yoes

Florian Hermann Froese

A technique for retaining a context in which data resides independently of a data store from which the data originates is disclosed. In relation to a method aspect of the technique, a computer-implemented method provides data with related first metadata, both originating from a data store and extracts the data and the related first metadata independently from the data store. A universal unique identifier of a portion of the data to which portion specific first metadata exists is created as part of the related first metadata. The universal unique identifier of the portion of the data is integrated into the related first metadata, thereby creating modified first metadata as an independently manageable and linkable representation of the related first metadata.

Preserving metadata context in a hybrid cloud context

The present disclosure relates to a control system for remotely controlling a change data capture (CDC) system. The CDC system comprises a source computing system and target computing system. The target computing system is configured to store a copy of data of the source computing system. The source computing system and the target computing system are configured to execute coordinated actions using predefined agents in order identify a change to data of the source computing system and to propagate, and store the change to the target computing system. The control system is configured for dynamically installing User-Defined Functions, UDF functions, in the source and target systems in order to control the agents to perform the predefined actions.

Remote control of a change data capture system

A computer-implemented method for transferring changes made to data from a data source to a data destination may be provided. The method involves using a message broker with streaming channels for transferring data changes from a data source to a data consumer. To determine if data has changed, a portion of data labeled with a key is sent through a hash function. A hash value with the corresponding key that has already been sent to the data consumer is retrieved from a streaming channel and compared to the newly created hash value. If the newly created hash value is different the data is sent to data consumers by the message broker via one streaming channel, while the new hash value is stored in the other streaming channel.

Universal streaming change data capture

The present disclosure relates to a method for processing data of a database system, the database system being configured to connect to a centralized cache shared with other database systems. The method includes determining a unique identifier of the content of a table of the database system using a common identification method of the database systems. A database command may be generated for processing data of the table. The command references the identifier. It may be determined if the cache comprises results of a same command with the same identifier. The command may be processed at the database system or the results of the command may be retrieved from the cache depending on whether the cache comprises the results.

Processing data of a database system

A computerized method for ingesting data from a relational database into a data lake is provided, wherein a user-defined function (UDF) is associated with a standard operation of extract, transform, load, or ETL, of an ETL pipeline. This UDF is triggered upon performing the standard operation and thereby allow a code associated with the UDF to be executed. Upon migrating data from one or more data sources into the relational database, the standard operation is executed, which triggers the UDF and, in turn, an execution of the code. As per the execution of this code, an entity running on the data lake is notified that a set of data migrated to the relational database is to be ingested according to given ingestion modalities specified by the code. Finally, the set of data can be ingested into the data lake according to the modalities. Related computer program products are also provided.

Computerized methods and programs for ingesting data from a relational database into a data lake

A method for controlling the storage of data among multiple regional storage centers coupled through a network in a global storage system is provided. The method includes steps of: defining at least one dataset comprising at least a subset of the data stored in the global storage system; defining at least one ruleset for determining where to store the dataset; obtaining information regarding a demand for the dataset through one or more data requesting entities operating in the global storage system; and determining, as a function of the ruleset, information regarding a location for storing the dataset among regional storage centers having available resources that reduces the total distance traversed by the dataset in serving at least a given one of the data requesting entities and/or reduces the latency of delivery of the dataset to the given one of the data requesting entities.

Method and apparatus for automated migration of data among storage centers

A classification system executing on one or more computer systems includes a processor and a memory coupled to the processor. The memory includes a discovery engine configured to navigate through non-volatile memory storage to discover an identity and location of one or more files in one or more computer storage systems by tracing the one or more files from file system mount points through file system objects and to disk objects. A classifier is configured to classify the one or more the files into a classification category. The one or more files are associated with the classification category and stored in at least one data structure. The one or more files are classified using an installed storage driver method. Methods are also provided.

System and method for the classification of storage

A classification system executing on one or more computer systems includes a processor and a memory coupled to the processor. The memory includes a discovery engine configured to navigate through non-volatile memory storage to discover an identity and location of one or more files in one or more computer storage systems by tracing the one or more files from file system mount points through file system objects and to disk objects. A classifier is configured to classify the one or more the files into a classification category. The one or more files are associated with the classification category and stored in at least one data structure. Methods are also provided.

Growing community of inventors

Wadenswil, Switzerland

Peter Urbanetz