International Business Machines Corporation

John G Rooney

Daniel Nikolaus Bauer

Luis Garcés-Erice

Paolo Scotton

David Clerc

Luis Garcés Erice

Jacques Fontignie

Christopher J Giblin

Peter Urbanetz

Paul T Hurley

Claudio Marinelli

Marcel Waldvogel

Florian Hermann Froese

Marc Vuilleumier Stückelberg

Pascal Vetsch

Ajay Dholakia

Peter Buhler

Thomas Gschwind

Tomas Tuma

Andreas Schade

Patrick Droz

Patrick M Stuedi

Embodiments of the present invention provide methods, computer program products, and systems. For each dataset in the system, embodiments of the present invention can identify at least one stack of hierarchically-dependent system components on which that dataset is dependent, and generate a system graph having nodes interconnected by directed edges. Dependency of each dataset on a stack, and between successive components in each stack, is represented by a directed edge between nodes of the graph. Embodiments of the present invention can then calculate for each dataset, a stack-dependent compliance score as a function of the compliance scores for a set of nodes reached by traversing directed edges in the graph from the node corresponding to that dataset, and generate an alert if the stack-dependent compliance score for a dataset breaches a predetermined risk threshold for that dataset.

Compliance risk management for data in computing systems

The present disclosure relates to a method for a cryptographic key rotation in a publish-subscribe system providing a broker service for routing stored encrypted messages to one or more subscribers of the topic to which the routed messages are assigned. The routing comprises decrypting the stored encrypted messages. The cryptographic key rotation comprises a re-encryption of the stored messages using a cryptographic replacement key. The re-encryption is executed by an encryption module of the publish-subscribe system as a background process, while the broker service is continued.

Key rotation on a publish-subscribe system

A technique for retaining a context in which data resides independently of a data store from which the data originates is disclosed. In relation to a method aspect of the technique, a computer-implemented method provides data with related first metadata, both originating from a data store and extracts the data and the related first metadata independently from the data store. A universal unique identifier of a portion of the data to which portion specific first metadata exists is created as part of the related first metadata. The universal unique identifier of the portion of the data is integrated into the related first metadata, thereby creating modified first metadata as an independently manageable and linkable representation of the related first metadata.

Preserving metadata context in a hybrid cloud context

The present disclosure relates to a control system for remotely controlling a change data capture (CDC) system. The CDC system comprises a source computing system and target computing system. The target computing system is configured to store a copy of data of the source computing system. The source computing system and the target computing system are configured to execute coordinated actions using predefined agents in order identify a change to data of the source computing system and to propagate, and store the change to the target computing system. The control system is configured for dynamically installing User-Defined Functions, UDF functions, in the source and target systems in order to control the agents to perform the predefined actions.

Remote control of a change data capture system

A method includes obtaining a first snapshot of a source system and deriving a set of key-value pairs reflecting the first snapshot. A mirror operation of the source system is performed to obtain CDC change operations. The method obtains a first CDC log as a first sequence of key-value pairs. A second snapshot of the source system is obtained and a set of key-value pairs is derived, which reflects the second snapshot. The first sequence of key-value pairs is compared with the set of key-value pairs to derive corrective CDC operations, captured as a set of key-value pairs. The corrective CDC operations represent corrections to be performed with respect to the first sequence of key-value pairs. A second CDC log is obtained as a second sequence of key-value pairs. The corrective CDC operations ensure that the second sequence of key-value pairs are coherent with the set of key-value pairs.

Tracking change data capture log history

An approach to finding data locations may be provided. A first synopses, which corresponds to first tabular data may be provided. An auxiliary data storage system may be scanned to identify second tabular data stored in the auxiliary data storage system. A second synopses may be obtained, in which the synopses correspond to the second columns of second tabular data. The synopsis may be computed for each second column of the second columns, according to a numeric representation of contents of cells of said each second column. The computed synopsis may include a vector of m descriptors. The two sets of one or more descriptors may be compared. A subset of the descriptors of the second synopses may be compared with corresponding descriptors of the first synopses, to identify potential matches between the second tabular data and the corpus of first tabular data.

Finding locations of tabular data across systems

A computer-implemented method for transferring changes made to data from a data source to a data destination may be provided. The method involves using a message broker with streaming channels for transferring data changes from a data source to a data consumer. To determine if data has changed, a portion of data labeled with a key is sent through a hash function. A hash value with the corresponding key that has already been sent to the data consumer is retrieved from a streaming channel and compared to the newly created hash value. If the newly created hash value is different the data is sent to data consumers by the message broker via one streaming channel, while the new hash value is stored in the other streaming channel.

Universal streaming change data capture

A server system includes a database for storing data, a memory device for storing data selected from the data in the database, and a processing device that selects data from the database, stores the selected data in the memory device, and transmits over a communication network an availability signal to a client, the availability signal including an identifier of the selected data and indicating that the selected data is stored in the memory device and is available to be accessed by the client.

Transmitting data including pieces of data

The present disclosure relates to a method for processing data of a database system, the database system being configured to connect to a centralized cache shared with other database systems. The method includes determining a unique identifier of the content of a table of the database system using a common identification method of the database systems. A database command may be generated for processing data of the table. The command references the identifier. It may be determined if the cache comprises results of a same command with the same identifier. The command may be processed at the database system or the results of the command may be retrieved from the cache depending on whether the cache comprises the results.

Processing data of a database system

A computer implemented method for enabling data coherency between a first data source and a second data source of different structure may be provided. The first data source has a first format and a related first log file. Similar conditions apply to the second data source. The method comprises also determining a first entry in the first log file at which the data have a semantic consistency and storing this as first read barrier. The same applies for an entry in the second data source. Additionally, the method comprises building a third data set derived from data of the first data source and data of the second data source and being coherent with them, using data from the first data source up to the first read barrier and data from the second data source up to the second read barrier.

Global coherency for distributed data sources

A computerized method for ingesting data from a relational database into a data lake is provided, wherein a user-defined function (UDF) is associated with a standard operation of extract, transform, load, or ETL, of an ETL pipeline. This UDF is triggered upon performing the standard operation and thereby allow a code associated with the UDF to be executed. Upon migrating data from one or more data sources into the relational database, the standard operation is executed, which triggers the UDF and, in turn, an execution of the code. As per the execution of this code, an entity running on the data lake is notified that a set of data migrated to the relational database is to be ingested according to given ingestion modalities specified by the code. Finally, the set of data can be ingested into the data lake according to the modalities. Related computer program products are also provided.

Computerized methods and programs for ingesting data from a relational database into a data lake

A computer implemented method includes storing data in a server, selecting a block of data from the stored data, storing the selected block of data in a cache memory of the server, and transmitting a block availability signal to a client, the block availability signal including an identifier of the selected block of data and indicating that the selected block of data is stored in the cache memory and is available to be accessed by the client.

Embodiments are directed to a computer-implemented method for ranking a proximity of data sources with one or more reference entities, which may be authoritative entities. The method comprises identifying several data sources and one or more reference entities. For each of the identified reference entities a vector of terms representative of said each of the reference entities is obtained. For each data source of the identified data sources, terms from said each data source are selected, an array comprising the selected terms can be accordingly populated, and a term frequency of terms of the vector obtained for said each of the reference entities is determined, in said array. The term frequency corresponds to the number of occurrences of said each of the terms in said array. The method ranks a proximity of the identified data sources with each of the one or more reference entities.

Ranking proximity of data sources with authoritative entities in social networks

Task scheduling in a hybrid cloud that includes a private cloud and an external cloud is performed. First a job to be performed is identified. Then, the job identified is decomposed into a set of tasks, whereby a task schedule is obtained for tasks of the set of tasks to be executed across the hybrid cloud. Next, a task to be executed in the external cloud that requires private data from the private cloud for its execution is detected from the task schedule. Finally, one or more non-anonymized portions of the private data are anonymized before execution of the detected task in the external cloud, by executing an anonymizing function from the private cloud. De-anonymization functionality may similarly be involved.

Task scheduling on hybrid clouds using anonymization

A method and system for transmitting data including pieces of data. The method includes the steps of: placing a piece of data on at least one cache memory; and sending a signal indicating a presence of the piece of data on the cache memory to at least one client, where at least one of the steps is carried out by a computer device.

Growing community of inventors

Zurich, Switzerland

John G Rooney