International Business Machines Corporation

Columbia University

Sybase, Inc.

Jun Rao

Guy Maring Lohman

Nimrod Megiddo

Mir Hamid Pirahesh

Calisto Paul Zuzarte

Sam Sampson Lightstone

Eugene Jon Shekita

Alfredo Alba

Eileen Tien Lin

Thomas D Griffin

Chad Eric DeLuca

Vuk Ercegovac

Daniele Costante Zilio

Latha S Colby

Chun Zhang

Sangeeta T Doraiswamy

Dengfeng Gao

Ihab F Ilyas

Kevin B Wang

Hetal Thakkar

Debabrata Dash

Asim V Singh

Adam J Storm

Bruce Gilbert Lindsay

Wei Fan

Kevin Scott Beyer

David Everett Simmen

Steven A Kirk

Kenneth Andrew Ross

Volker Gerhard Markl

William T O'Connell

Randall G Bello

Sandeep Tata

Ivan Popivanov

Roger D MacNicol

Julia Stoyanovich

Robin D Van Boeschoten

Katherine Tsouya Yang

A method for creating a semantically aggregated index in an indexer-agnostic index building system includes: extracting documents from a data source, each document including a data object; distributing the documents to a plurality of processing nodes within the system; for each node: indexing the data objects for each document into fields using semantic rules; and grouping indexed data objects for related fields by: classifying the documents into logical groups based on the semantic rules; and creating a searchable index shard for related logical groups.

Creating a semantically aggregated index in an indexer-agnostic index building system

A computer program product for an indexer-agnostic index building system includes a computer readable storage medium to store a computer readable program, wherein the computer readable program, when executed on a computer, causes the computer to perform operations for creating a semantically aggregated index. The operations include: extracting documents from a data source, wherein each document includes a data object; distributing the documents to a plurality of processing nodes within the system; for each node: indexing the data objects for each document into fields using semantic rules; and grouping indexed data objects for related fields by: classifying the documents into logical groups based on the semantic rules; and creating a searchable index shard for related logical groups.

Semantically aggregated index in an indexer-agnostic index building system

A method for updating a scalable row-store, including: receiving an update to a key within a range of keys in a database table, wherein the database table is distributed across nodes in a cluster of computing devices; and replicating the update over a group of the nodes using a consensus-based replication algorithm, wherein the replication algorithm includes completing the update in response to receiving acknowledgement messages from a majority of the nodes in the group indicating that the majority has received notification of the update.

Scalable row-store with consensus-based replication

Disclosed is a method of encoding JavaScript Object Notation (JSON) documents in an inverted index, wherein a tree representation of a JSON document is first generated, and, next, the JSON document is shredded into a list of &#x3c;value, path, type, jdewey&#x3e; tuples for each atom node, n, in the tree, where value is a label associated with n, path is a concatenation of node labels associated with ancestors of n, type is a description of a type of value, and jdewey of n is a partial Dewey code of its closest ancestor array node, if one exists, or empty, otherwise. Lastly, an inverted index is built using &#x3c;path, type, value&#x3e; as index term, and jdewey as payload. A method is also described to search the inverted index.

Indexing and searching JSON objects

A method and system for cleansing anomalies from sequence-based data at query time. Sequence-based data such as RFID data is loaded into a database. One or more cleansing rules are received at a cleansing rules engine. The cleansing rule engine converts the cleansing rule(s) to a template that includes logic to compensate for anomalies in the sequence-based data. A query to retrieve the sequence-based data is received by a query rewrite engine. The query rewrite engine rewrites the query by applying the template logic. The rewritten query is executed at query time. The result of the rewritten query execution is identical to the result of executing the original query on a data set generated by applying the cleansing rule to all of the sequence-based data.

Method and system for cleansing sequence-based data at query time

A system for automating data partitioning in a parallel database includes plural nodes connected in parallel. Each node includes a database server and two databases connected thereto. Each database server includes a query optimizer. Moreover, a partitioning advisor communicates with the database server and the query optimizer. The query optimizer and the partitioning advisor include a program for recommending and evaluating data table partitions that are useful for processing a workload of query statements. The data table partitions are recommended and evaluated without requiring the data tables to be physically repartitioned.

System and method for automating data partitioning in a parallel database

A method and system for automatically and adaptively determining query execution plans for parametric queries. A first classifier trained by an initial set of training points is generated. A query workload and/or database statistics are dynamically updated. A new set of training points is collected off-line. Using the new set of training points, the first classifier is modified into a second classifier. A database query is received at a runtime subsequent to the off-line phase. The query includes predicates having parameter markers bound to actual values. The predicates are associated with selectivities. A mapping of the selectivities into a plan determines the query execution plan. The determined query execution plan is included in an augmented set of training points, where the augmented set includes the initial set and the new set.

Automatically and adaptively determining execution plans for queries with parameter markers

A compilation time estimator provides a quantified estimate of the optimizer compilation time for a given query optimizer. The estimator automates the optimizer to choose the right level of optimization in commercial database systems. The estimator reuses an optimizer's join enumerator to obtain actual number of joins, but bypasses plan generation to save estimation overhead, and maintains a small number of interesting physical properties to estimate the number of plans by using a linear regression model. The estimator uses the number of generated plans to estimate query compilation time.

Estimating the compilation time of a query optimizer

A method for cleansing anomalies from sequence-based data at query time. Sequence-based data such as RFID data is loaded into a database. A cleansing rule is received at a cleansing rules engine. The cleansing rule engine converts the cleansing rule to a template that includes logic to compensate for anomalies in the sequence-based data. A query to retrieve the sequence-based data is received by a query rewrite engine. The query rewrite engine rewrites the query by applying the template logic and an expanded rewrite technique. The rewritten query is executed at query time. The result of the rewritten query execution is identical to the result of executing the original query on a data set generated by applying the cleansing rule to all of the sequence-based data.

Method for cleansing sequence-based data at query time

Exemplary embodiments of the present invention relate to enhanced faceted search support for OLAP queries over unstructured text as well as structured dimensions by the dynamic and automatic discovery of dimensions that are determined to be most 'interesting' to a user based upon the data. Within the exemplary embodiments 'interestingness' is defined as how surprising a summary along some dimensions is from a user's expectation. Further, multi-attribute facets are determined and a user is optionally permitted to specify the distribution of values that she expects, and/or the distance metric by which actual and expected distributions are to be compared.

Computer automated discovery of interestingness in faceted search

A system, apparatus, and program storage device implementing a method of optimizing queries used for searching a computerized database, wherein the method comprises providing a query comprising a sequence of inner joins and outerjoins; and rewriting the query by producing a sequence of outer Cartesian products for the query; producing a sequence of nullification operations on the query; and producing a sequence of best match operations on the query. The method further comprises optimizing the query using a query execution plan for processing the rewritten query, wherein the query execution plan expands a search space in the database for which the rewritten query may be run.

Canonical abstraction for outerjoin optimization

Disclosed is a method for directing a data processing system, the data processing system and an article of manufacture. The data processing system has memory for storing a database. The method directs the data processing system to configuring the database based on a database workload having a performance metric. The method includes selecting a plurality of database characteristics based upon the database workload, each of the database characteristics having an associated estimated performance metric for the database workload, selecting a set of database characteristics from the plurality of database characteristics for which the associated estimated performance metric is lower than the database workload performance metric, and configuring the database using the selected set of database characteristics.

Method, system and program for selection of database characteristics

Discovering interestingness in faceted search

Growing community of inventors

San Jose, CA, United States of America

Jun Rao