International Business Machines Corporation

Workday, Inc.

Paraccel LLC

Mayank Pradhan

Edison Lao Ting

Dung Kim Nguyen

Kevin Scott Beyer

Tuong Chanh Truong

Tian Zhang

Anjali Betawadkar-Norwood

Normen Seemann

Vignesh Sukumar

Radhika Reddy

Robbert C Van Der Linden

David G Wilhite, Jr

Adam Birnbaum

Raymi Dyskant

David Benedict Galimberti

Brian Pak-Ning Chu

Systems and methods for optimizing memory in an interest-driven business intelligence system in accordance with embodiments of the invention are illustrated. A dictionary for storing values of a dataset may be partitioned in accordance with some embodiments. The partitions of the dictionary may be generated by mapping and reducer processes. The mapping processes receive a value, determine the dimension of the data to which the value belongs, and provides the value to a reducer process that handles the determined dimension. Each reducer process generates partitions of the dictionary for each dimension. The number of values in each partition is determined and compared to a threshold value. Partitions that have a number of values greater than the threshold are stored in a common memory. Partitions smaller than the threshold value can be combined with other partitions such that the cardinality of the combined partition exceeds the threshold value.

Systems and methods for memory optimization interest-driven business intelligence systems

Embodiments are included for methods, systems, and computer program products in which evaluating operations using an electronic computing device are presented including: receiving the operation on a first database partition in a shared nothing parallelism system, where the operation is non-collocated, reference based operation; generating a correlation sequence, where the correlation sequence includes a sequence of references pointing to input data required for the operation; receiving one of the references on a first table queue operator on a second database partition, where the second database partition includes input data corresponding with the received reference, and where the table queue operator is configured for providing communication between the first database partition and the second database partition; and processing the input data on the second database partition corresponding with the operation.

Evaluating reference based operations in shared nothing parallelism systems

A database system performs hash join process for processing queries that join an inner and an outer database table. The hash join processes builds a hash table in memory for the inner table. The database system receives a limit on the memory for storing the hash table. The database system maximizes the number of partitions stored in memory for the hash table. If the hash table exceeds the limit of the memory while adding rows from the inner table, the database system selects a partition for spilling to a persistent storage. The partition selected for spilling to may be the largest partition or a partition larger than most of the partitions. The database system initializes the hash table to a number of partitions that is substantially equal to half of the total number of blocks that can be stored within the specified limit of memory for the hash table.

Disk-based hash join process

Embodiments are included for methods, systems, and computer program products for evaluating operations in a shared nothing parallelism system using an electronic computing device are presented, the methods and computer program products including: causing the electronic computing device to receive the operation, where the operation is non-collocated, reference based operation, and where the operation is configured for execution on a first partition in the shared nothing parallelism system; causing the electronic computing device to insert a table queue operator on a second partition, where the second partition includes a reference to input data, causing the electronic computing device to place the reference to the input data on the first partition by the table queue operator; and causing the electronic computing device to execute the operation on the reference to the input data on the first partition.

A method and system for distributed garbage collection in a pipelined workflow environment comprising a plurality of processing nodes that pass item references to each other. Each node maintains a reference list of local item references and a reference dictionary of remote item references, and periodically synchronizes the reference list with the reference dictionaries of other nodes in the workflow, so that item references are not marked for garbage collection while other processing nodes may still have outstanding references to them.

Distributed garbage collection in a dataflow system

A system and computer-based method is provided for context-sensitive decomposition of a markup based document into a relational database, based on schemas with reusable item declarations. The user creates a mapping document from a schema of a markup based document with at least one reusable item declaration defining an item being mapped to different tables/columns pairs, depending on the item's ancestry. Mapping document is then annotated with the item ancestry mapping annotation having a location path of the item. Each item is decomposed into a corresponding table column. Preferably, the mapping document is an annotated XML Schema.

System and method for context-sensitive decomposition of XML documents based on schemas with reusable element/attribute declarations

Methods for pre-processing mapping information for efficient decomposition of an XML document for storage in a database are provided. The method includes receiving a mapping document that describes how all of (or a portion of) an XML document is to be decomposed, transforming the mapping document into a data structure for decomposing an XML document, and making the data persistent for use with a subsequent decomposition operation that decomposes an XML document.

Method for pre-processing mapping information for efficient decomposition of XML documents

A system and computer-based method is provided for decomposing and storing a markup based document into a relational database. For a schema of a markup based document a user identifies multiple items mapping into a same database table-column pair, creates a logical table and associates the logical table to each item and a corresponding database table. Next, a user creates a mapping document of the markup based document with mapping annotations defining mapping of the items into columns of the logical tables. Decomposition of each item into a corresponding logical table column is accomplished by collecting the item content from the markup based document and storing it in the corresponding row column.

Growing community of inventors

San Jose, CA, United States of America

Mayank Pradhan