International Business Machines Corporation

Green Market Square Limited

Pedro Miguel Barbas

David M Tilson

Konrad Emanowicz

Joseph Duffy

Austin Clifford

Ken Maycock

Lei Pan

David Kelly

Enda McCallig

Enzo Cialini

Gareth Jenkins

Patrick G O'Sullivan

Clara Liu

Nitin Kumar Gupta

Hima Patel

Shaikh Shahriar Quader

Aslam F Nomani

Ilker Ender

Sami Abed

Mara Elisa De Paiva Fernandes Matias

Hemant Asandas Bhatia

Peter J Duffy

Manohar Bashyam

Adrian Mahjour

Gary Murtagh

Deepak Kulkarni

Rodrigo Cravo Dorea Arnez

Guilherme Rodrigues De Abreu

Martin J Neary

Christian Cesar Bones

Johnson Uman

Paddy Burke

Brian McKeown

Donald Fraser

Jan Mahon

Qinlong Luo

A method, computer program product, and computer system for clustering data objects. Data objects are accessed. The data objects are sorted. The data objects are transformed into binary words. The binary words are encoded into blocks, using the sorted data objects. Block clusters are generated from the blocks. The block clusters are converted into word clusters. For each word cluster, the word cluster is reconfigured into L word clusters in a manner that minimizes a total number of binary word deviations in the L word clusters, wherein L is at least 1.

Privacy preserving and high performance data clustering

A cleansing operation defined for a data structure of a database managed by a database management system is obtained. The cleansing operation is performed on data of the data structure to obtain clean data. The cleansing operation that is defined for the data structure and performed on data of the data structure is performed by the database management system.

In-database data cleansing

A cleansing operation is performed on data of a data structure to obtain clean data. The clean data is stored as part of the data structure; however, the clean data is independent of the data. A mapping is performed to provide a set of ordered data that includes the data and the clean data.

In-database data cleansing and independent store of clean data

A computer-implemented method, system and computer program product for optimally performing stress testing against big data management systems. A set of random test queries is generated and compiled to determine the data points of the features (e.g., table type being queried) of the set of random test queries. A distance (e.g., Mahalanobis distance) is then measured between the data points of the features and the mean of a distribution of data points corresponding to each same feature of an extracted feature set. Each random test query whose distance exceeds a threshold distance is then ranked. The ranked random test queries are then executed in order of rank. Those executed random test queries which resulted in an error (e.g., system failure) are added to a log, which is used to identify those queries to perform a stress test against the big data management system.

Workload generation for optimal stress testing of big data management systems

A method of accessing objects with fine-grained access control (FGAC) in a relational database management system (RDBMS) storing a segmented column-major database. For each object with access restrictions, an artificial neural network (ANN), is trained by generating an equally distributed segment map of segmented data entries, so that the map reproduces the row disposition in the unsegmented object. When a user access request is received, these ANNs are referred to determine if any of the objects to be accessed are subject to access restrictions. If that is the case, then the ANN creates a pseudo-view construct of its associated object which is limited to data entries that the user has permission to access. The pseudo-views are then injected into the user access request to embed the fine-grained access controls for subsequent processing of the request, which can then proceed without further regard to user-specific access restrictions.

Fine-grained access control of column-major relational database management systems

Concepts for processing a SQL statement comprising a ROLAP sort operation are presented. One such concept comprises: determining whether a ROLAP partitioning column of the SQL statement does not refer to database columns from a sort operation of the SQL statement; and determining whether the ROLAP partitioning columns of the SQL statement are all referenced in the output of the SQL statement. Responsive to determining a ROLAP partitioning column of the SQL statement does not refer to database columns from a sort operation of the SQL statement, and to determining all of the ROLAP partitioning columns of the SQL statement are referenced in the output of the SQL statement, the SQL statement is modified by removing the ROLAP sort operation from the SQL statement.

Optimizing relational online analytical processing sort operations

Embodiments of the present invention provide systems and methods for associating rule mining in data comprising first and second variables, by generating a first and second filtering bitmap. The first filtering bitmap represents a set of values for the second variable for each of a plurality of values of the first variable. The second filtering bitmap represents, associations between frequent values of the second variable, which enables the identification of frequent 2-pattern itemsets. Thus, by performing first logical operations on the frequent 2-pattern itemsets, frequent high order-pattern itemsets may be identified. A max pattern itemset may be identified among the frequent high order-pattern itemsets. As a result, embodiments may mitigate performance and stability problems associated with conventional association rule mining approaches.

Association rule mining using max pattern transactions

Embodiments of the present invention provide systems and methods for associating rule mining. By way of example, embodiments obviate or mitigate performance and stability problems associated with conventional association rule mining approaches, by proposing concepts implementing association rule mining using rolling filtering bitmaps over dynamic data.

Association rule mining

A computer-implemented method, computer program product and system for identifying pseudonymized data within data sources. One or more data repositories within one or more of the data sources are selected. One or more privacy data models are provided, where each of the privacy data models includes pattern(s) and/or parameter(s). One or more of the one or more privacy data models are selected. Data identification information is generated, where the data identification information indicates a presence or absence of pseudonymized data and of non-pseudonymized data within the one or more of the data sources. The data identification information is generated utilizing the pattern(s) and/or the parameter(s) to determine pseudonymized data.

Runtime control of automation accuracy using adjustable thresholds

A method, system and computer program for implementing fine-grained access control (FGAC) of data stored in a dataset. In response to receiving a data query statement from a user, any representational index exclusions that are relevant to the data query statement are identified, wherein each index exclusion specifies an access restriction to the data. It is then determined whether any of the identified representational index exclusions are to be applied to the data query statement, and if 'yes' then the data query statement is modified before being processed, so that processing of the query takes place by searching the dataset under restriction of the representational index exclusions. The proposed approach allows for easy creation and modification of FGAC privacy rules without introducing performance gaps in processing the data query statements.

Fine-grained access control to datasets

As disclosed herein a computer system for secure database backup and recovery in a secure database network has N distributed data nodes. The computer system includes program instructions that include instructions to receive a database backup file, fragment the file using a fragment engine, and associate each fragment with one node, where the fragment is not stored on the associated node. The program instructions further include instructions to encrypt each fragment using a first encryption key, and store, randomly, encrypted fragments on the distributed data nodes. The program instructions further include instructions to retrieve the encrypted fragments, decrypt the encrypted fragments using the first encryption key, re-encrypt the decrypted fragments using a different encryption key, and store, randomly, the re-encrypted fragments on the distributed data nodes. A computer program product and method corresponding to the above computer system are also disclosed herein.

Secure database backup and recovery

Disclosed is an approach comprising a column partitioned into a plurality of partitions including an empty partition and a plurality of filled partitions each comprising data entries associated with a set of parameters having parameter values, the data entries compressed in accordance with a compression dictionary. The approach comprises receiving forecasted parameter values for an expected set of data entries to be stored in an empty partition; predicting a recurrence frequency of the data entries in the expected set using the forecasted parameter values by evaluating the respective compression dictionaries of the filled partitions with a machine learning algorithm; generating a predictive compression dictionary for the expected set of data entries based on the predicted recurrence frequency of the data entries in the expected set; receiving the expected set of data entries; and compressing at least part of the received expected set of data entries using the predictive compression dictionary.

Columnar database compression

As disclosed herein, a method includes receiving a plurality of datasets from a database, wherein each dataset comprises one or more data fields represented in a single data format, and wherein the data fields from at least two of the datasets are represented in different data formats, combining the plurality of datasets to provide a created data column corresponding to all of the data fields from the plurality of datasets, organizing the data column into data clusters, wherein each data cluster includes data fields represented in a single data format, and wherein each data field belongs to a data cluster, providing a key-value map referencing data fields with respect to their corresponding data formats, and verifying the database with respect to the created column. A corresponding computer program product and computer system are also disclosed.

Growing community of inventors

Dunboyne, Ireland

Pedro Miguel Barbas