International Business Machines Corporation

Marcin Filip

Michal Bodziony

Marcin Luczynski

Tomasz Zatorski

Andrzej W Laskawiec

Monika Piatek

Lukasz Stanislaw Studzienny

Bartosz Tomasik

Wojciech Mis

Lukasz Grzegorz Cmielowski

Andrzej Jan Wrobel

Mateusz S Matejczyk

A computer-implemented method, system and computer program product for creating virtualized data assets. A copy of an extract, transform, load (ETL) or an extract, load, transform (ELT) job definition is created. Furthermore, a table function is created in which the table function is configured to trigger the execution of a copy of the ETL/ELT job definition, to read the resulting data from executing the copy of the ETL/ELT job definition and to return the resulting data (data sets), such as in a queue in a federation server. After receiving a query from a user based on a virtual view in which the query includes the ETL/ELT job definition, the table function is executed which triggers execution of the copy of the ETL/ELT job definition. Data sets are then produced by executing the copied ETL/ELT job definition, which is read and returned to the user in the virtual view.

Creating virtualized data assets using existing definitions of ETL/ELT jobs

Providing an automatic mechanism of invalidating false-positive indications of certain identified portions of source code to reduce the number of errors in a security report. Certain embodiments of the present invention utilize static security scanning as a mechanism for automatically determining which portions of the identified source code contain potential vulnerabilities, and whether these identified portions of the source code are correctly or incorrectly identified with a false-positive indication.

False-positives invalidation and static security scans without scanning based on regular scan history in pull requests

Methods, computer program products, and systems are presented. The method computer program products, and systems can include, for instance: running an application on one or more computing node, the application providing user access to a database; receiving by the application registration data from a user for registering the user into a service; in response to the receiving the registration data from the user, generating a user schema that defines access rights of the user to the database, wherein the generated user schema includes a set of shared data structure objects instantiated in a storage system associated to the application, and a set of private data structure objects.

Dynamic schema based multitenancy

A computer-implemented method for searching for a similar image in an image database includes receiving a query image, computing a value of a similarity measure of the query image, comparing the value of the similarity measure of the query image with metadata of the partitions, discarding at least one partition of the image database based on the comparison of the value of the similarity measure of the query image with the metadata, comparing the images of at least one of the remaining partitions with the query image, and selecting at least one image of the at least one partition of the remaining partitions as the similar image based on the comparison of the images of the at least one of the remaining partitions with the query image.

Method for searching similar images in an image database using global values of a similarity measure for discarding partitions of the image database

Utilizing an Information Analyzer to profile data in order to identify data assets that contain executable code for the purpose of ensuring the security and integrity of the profiled data. The results of the data profiling process can be used by security policies to reduce the risks of malicious code execution attacks.

Detecting potentially malicious code in data through data profiling with an information analyzer

A computer-implemented method for unambiguously identifying entities in a database system may be provided. The method comprises storing data items as records with different attributes in a table of a database, storing naming rules for selected combinations of the attributes of the data items, and prioritizing the naming rules. The method also comprises determining a hash value for each of the selected combinations of the attributes of the data items, and identifying duplicate data items using the determined hash values and the prioritized naming rules.

Multi-value primary keys for plurality of unique identifiers of entities

A method, computer system, and a computer program product for ensuring integrity of records in a NoSQL database including a first table and a second table is provided. The present invention may include the first table having first records representing respective first entities and the second table having second records representing respective second entities. The present invention may include using a hash table associating each second entity of the second table with the respective hash or summary hash values of first records for reading the second records of the second table.

Ensuring integrity of records in a not only structured query language database

Managing ETL (extract-transform-load) operation execution by receiving an updated ETL (extract-transform-load) job flow, determining affected stages of a current ETL job flow, sending a message to stop data flow for all stages of the current ETL job flow, sending a switch message along the current ETL job flow through the affected stages, receiving notice of receipt of the switch message downstream from the affected stages of the current ETL job flow, moving an affected stage of the current ETL job flow from a first ETL section to a second ETL section; and resuming data flow according to the updated ETL job flow.

Managing extract-transform-load operations

Aspects of the present invention disclose a method for ordering rows of a table based on access statistics. The method includes one or more processors adding a respective tracking mechanism to the attribute of each respective instance of the set of rows of the table of the database. The method further includes adding a tracking mechanism to the respective instances of the set of rows. The method further includes gathering access data of the respective tracking mechanism of the attribute of each respective instance of the set of rows. The method further includes determining each respective instance of an attribute value that corresponds to the access data of the respective tracking mechanism of each respective instance of the set of rows. The method further includes ordering each respective instance of the set of rows of the table based at least in part on the respective instances of the attribute values.

Indexing data in a table based on data usage statistics

A method of masking data includes disabling, by one or more processors of a computer system, operation of a data structure that includes at least one data index. The method includes masking, by the one or more processors of the computer system, the data structure in a table with an isomorphic function, and enabling, by the one or more processors of the computer system, operation of the data structure.

Data masking with isomorphic functions

A method, computer program product, and a system where a processor(s) intercepts a query from a client targeting a database. The processor(s) analyzes the query to determine contextual data relevant to the query and determines, based on a data masking policy, if a portion of results responsive to the intercepted query should be provided to the client as masked data. When the processor(s) determines that the portion of the results should be provided to the client as masked data, the processor(s) determines computing resources managing data comprising the results responsive to the intercepted query. The processor(s) executes the query on the computing resources by obtaining the masked data via coprocessors deployed on the computing resources, where the coprocessors mask the portion of the results. The processor(s) returns query results comprising the portion of the results masked by the coprocessors, to the client.

Push down policy enforcement

An approach is provided for encrypting data. Using an encryption function, values of keys in a first database table are encrypted. The encryption function is determined to be homomorphic to sorting operators. A decryption function that decrypts the encrypted keys is determined to be homomorphic to sorting operators. Responsive to the encryption and decryption functions being determined to be homomorphic, a merge join operation is selected. The merge join operation operates on the first database table and a second database table and includes the decryption function in a joining condition. Using the merge join operation, an execution of a query is optimized. The query accesses one or more data items in the first or second database table.

Enable merge join on encrypted data

Methods and systems for moving a dataset from a relational database to a non-relational database are disclosed. A method includes: copying, by a computing device, source datafiles from a relational database to each of a plurality of region servers associated with a non-relational database; generating, by the computing device, at least one table in the non-relational database from the copied source datafiles; and accessing, by the computing device, the at least one table in the non-relational database by reading raw data stored in a relational database format from the copied source datafiles on one of the plurality of region servers associated with the non-relational database.

Non-relational database coprocessor for reading raw data files copied from relational databases

A computer-implemented method includes: receiving, by a computer device, a database query of a database from a database client; determining, by the computer device and based on the database query, a to-be-masked portion of data; sending, by the computer device, the database query to a federated database system for the to-be-masked portion of the data; receiving, by the computer device and in response to the sending, masked resultant data from the federated database system; and forwarding, by the computer device, the masked resultant data to the database client. The to-be-masked portion of the data is masked in at least one view of the database residing on the federated database system.

Redirecting query to view masked data via federation table

Optimization of job execution includes generating a hash tree for an initial dataset, executing a data processing job against the initial dataset, the job including a set of calculations against data of the records of the initial dataset, generating, based on a modification to the initial dataset to produce a modified dataset, a hash tree for the modified dataset, comparing the hash tree for the initial dataset to the hash tree for the modified dataset, and identifying one or more blocks of the modified dataset that are updated compared to the initial dataset, and re-executing the job against the modified dataset. The re-executing performs only those calculations that use data of the records of the one or more blocks that are updated compared to the initial dataset, and does not perform other calculations that do not use data of the records of the one or more blocks.

Growing community of inventors

Cracow, Poland

Marcin Filip