International Business Machines Corporation

Michal Bodziony

Artur M Gruszecki

Tomasz Kazalski

Marcin Filip

Konrad K Skibski

Marcin Luczynski

Tomasz Zatorski

Lukasz Gaza

Andrzej W Laskawiec

Monika Piatek

Lukasz Stanislaw Studzienny

Rafal Korczyk

Bartosz Tomasik

Wojciech Mis

Andreas Brodt

Grzegorz Piotr Szczepanik

Lukasz Jakub Palus

Krzysztof A Rudek

Tomasz Stradomski

Tomasz Sekman

Krzysztof Anton

Pawel K Koperek

Torsten Steinbach

Stefanie Scherzinger

Andrzej Laskawiec

Lukasz Studzienny

A computer-implemented method, system and computer program product for creating virtualized data assets. A copy of an extract, transform, load (ETL) or an extract, load, transform (ELT) job definition is created. Furthermore, a table function is created in which the table function is configured to trigger the execution of a copy of the ETL/ELT job definition, to read the resulting data from executing the copy of the ETL/ELT job definition and to return the resulting data (data sets), such as in a queue in a federation server. After receiving a query from a user based on a virtual view in which the query includes the ETL/ELT job definition, the table function is executed which triggers execution of the copy of the ETL/ELT job definition. Data sets are then produced by executing the copied ETL/ELT job definition, which is read and returned to the user in the virtual view.

Creating virtualized data assets using existing definitions of ETL/ELT jobs

Providing an automatic mechanism of invalidating false-positive indications of certain identified portions of source code to reduce the number of errors in a security report. Certain embodiments of the present invention utilize static security scanning as a mechanism for automatically determining which portions of the identified source code contain potential vulnerabilities, and whether these identified portions of the source code are correctly or incorrectly identified with a false-positive indication.

False-positives invalidation and static security scans without scanning based on regular scan history in pull requests

Methods, computer program products, and systems are presented. The method computer program products, and systems can include, for instance: running an application on one or more computing node, the application providing user access to a database; receiving by the application registration data from a user for registering the user into a service; in response to the receiving the registration data from the user, generating a user schema that defines access rights of the user to the database, wherein the generated user schema includes a set of shared data structure objects instantiated in a storage system associated to the application, and a set of private data structure objects.

Dynamic schema based multitenancy

A computer-implemented method for searching for a similar image in an image database includes receiving a query image, computing a value of a similarity measure of the query image, comparing the value of the similarity measure of the query image with metadata of the partitions, discarding at least one partition of the image database based on the comparison of the value of the similarity measure of the query image with the metadata, comparing the images of at least one of the remaining partitions with the query image, and selecting at least one image of the at least one partition of the remaining partitions as the similar image based on the comparison of the images of the at least one of the remaining partitions with the query image.

Method for searching similar images in an image database using global values of a similarity measure for discarding partitions of the image database

Utilizing an Information Analyzer to profile data in order to identify data assets that contain executable code for the purpose of ensuring the security and integrity of the profiled data. The results of the data profiling process can be used by security policies to reduce the risks of malicious code execution attacks.

Detecting potentially malicious code in data through data profiling with an information analyzer

A computer-implemented method for unambiguously identifying entities in a database system may be provided. The method comprises storing data items as records with different attributes in a table of a database, storing naming rules for selected combinations of the attributes of the data items, and prioritizing the naming rules. The method also comprises determining a hash value for each of the selected combinations of the attributes of the data items, and identifying duplicate data items using the determined hash values and the prioritized naming rules.

Multi-value primary keys for plurality of unique identifiers of entities

A method, computer system, and a computer program product for ensuring integrity of records in a NoSQL database including a first table and a second table is provided. The present invention may include the first table having first records representing respective first entities and the second table having second records representing respective second entities. The present invention may include using a hash table associating each second entity of the second table with the respective hash or summary hash values of first records for reading the second records of the second table.

Ensuring integrity of records in a not only structured query language database

Managing ETL (extract-transform-load) operation execution by receiving an updated ETL (extract-transform-load) job flow, determining affected stages of a current ETL job flow, sending a message to stop data flow for all stages of the current ETL job flow, sending a switch message along the current ETL job flow through the affected stages, receiving notice of receipt of the switch message downstream from the affected stages of the current ETL job flow, moving an affected stage of the current ETL job flow from a first ETL section to a second ETL section; and resuming data flow according to the updated ETL job flow.

Managing extract-transform-load operations

In an approach for migrating database content with row-level security from a source database with row-level security to a target database without row-level security, a computer determines a type of the source database to be migrated, and upon determining that a source table of the source database is maintained using multi-level security attributes, creating a data structure for a source table of the source database for storing meta-data comprising the multi-level security attributes, determining all dimensions of the multi-level security attributes, adding columns to a target table of the target database relating to the source table, the added columns representing the multi-level security attributes of the source table.

Migrating a multi-level secured database

A computer-implemented method, system and computer program product for applying a unified governance and integration platform to social media data. Data integration definitions for managing and protecting social media data are received. After receiving the data integration definitions for social media data, data integration externalization, governance catalog externalization or lineage externalization may be performed. For example, social media data may be extracted from a database system by a governance and integration unit ('unit') using the data integration definitions. The extracted social media data is then transformed and loaded to the social media data system via a connector linking the data from the unit to the social media data system. In another example, social media data is extracted from the social media data system by the unit, such as via the connector, using the data integration definitions and then transformed and loaded to the database system for analysis.

Applying unified governance and integration platform to social media data

Automatically extracting lineage data for distributed file systems and databases by generating a file folder including data and having a file path, generating a data table associated with the data and the file path, tracking the lineage of data into the file folder and the lineage of data from the data table, and recording an overall lineage comprising the lineage of data into the file folder and from the data table.

Automatically capturing lineage data in distributed systems

Aspects of the present invention disclose a method for ordering rows of a table based on access statistics. The method includes one or more processors adding a respective tracking mechanism to the attribute of each respective instance of the set of rows of the table of the database. The method further includes adding a tracking mechanism to the respective instances of the set of rows. The method further includes gathering access data of the respective tracking mechanism of the attribute of each respective instance of the set of rows. The method further includes determining each respective instance of an attribute value that corresponds to the access data of the respective tracking mechanism of each respective instance of the set of rows. The method further includes ordering each respective instance of the set of rows of the table based at least in part on the respective instances of the attribute values.

Indexing data in a table based on data usage statistics

A computer-implemented method for joining two data tables on a join attribute, where the data tables have at least a first and a second attribute and the second attribute is the join attribute. The method provides a function for associating a computing node to a given record. The function may be used to determine the associated computing node. The records of the two data tables may be distributed to the respective determined computing nodes. The relationship between the values of the first and second attributes may be modelled using a predefined dataset. For each record of the two data tables the values of the first attribute may be re-determined using the corresponding values of the second attribute. The function may be used to re-determine the associated computing node.

Joining two data tables on a join attribute

A method of masking data includes disabling, by one or more processors of a computer system, operation of a data structure that includes at least one data index. The method includes masking, by the one or more processors of the computer system, the data structure in a table with an isomorphic function, and enabling, by the one or more processors of the computer system, operation of the data structure.

Data masking with isomorphic functions

A method, computer program product, and a system where a processor(s) intercepts a query from a client targeting a database. The processor(s) analyzes the query to determine contextual data relevant to the query and determines, based on a data masking policy, if a portion of results responsive to the intercepted query should be provided to the client as masked data. When the processor(s) determines that the portion of the results should be provided to the client as masked data, the processor(s) determines computing resources managing data comprising the results responsive to the intercepted query. The processor(s) executes the query on the computing resources by obtaining the masked data via coprocessors deployed on the computing resources, where the coprocessors mask the portion of the results. The processor(s) returns query results comprising the portion of the results masked by the coprocessors, to the client.

Growing community of inventors

Tegoborze, Poland

Michal Bodziony