Sap Se

Microsoft Technology Licensing, LLC

Guido Moerkotte

Norman May

Alexander Boehm

Robert Schulze

Michael Brendle

Michael Grossniklaus

Wolfgang Lehner

Lars Fricke

Florian Wolf

Axel Hertzschuch

Siomara Schulz

Per-Ake Larson

Jingren Zhou

Peter Lysakovski

Diego Havenstein

Gabriele Steidl

Frank W Tompa

Mahammad Valiyev

Nick Weber

Provided are systems and methods for creating histograms with distinct value sketches integrated therein and for query processing based on the histograms with distinct value sketches. In one example, the method may include storing a histogram that comprises a representation of a bucket of data from a database and that includes a distinct value sketch with a distinct value attribute that identifies an estimated number of distinct values within the bucket of data, receiving a database query, generating a query execution plan for the database query based on the distinct value attribute of the bucket within the distinct value sketch embedded within the histogram, and executing the database query on the bucket of data from the database based on the generated query execution plan.

Histogram with integrated distinct value sketches

According to some embodiments, methods and systems may be associated with a cloud computing environment. A workload prediction framework may receive observed workload information associated with a database in the cloud computing environment (e.g., a DataBase as a Service ('DBaaS')). Based on the observed workload information, a Statement Arrival Rate (“SAR”) prediction may be generated. In addition, a host variable assignment prediction may be generated based on the observed workload information. The workload prediction framework may then use the SAR prediction and the host variable assignment prediction to automatically create a workload prediction for the database. A physical database design advisor (e.g., a table partitioning advisor) may receive the workload prediction and, responsive to the workload prediction, automatically generate a recommended physical layout for the database (e.g., using a cost model, the current physical layout, and an objective function).

Framework for workload prediction and physical database design

A system includes determination of a query comprising z predicates, determination of known selectivities of each of a set of conjuncts of the z predicates, and estimation of selectivities of all conjuncts of the z predicates based on the known selectivities and a recursive determination of Cx, where x represents selectivities of all complete conjuncts of the z predicates and C is a complete design matrix complete design matrix (c)=1 if i⊆j, else 0, having indices [0, 2−1].

Selectivity estimation of conjunctive predicates

The present disclosure involves systems, software, and computer implemented methods for improved design and implementation of data access metrics for automated physical database design. An example method includes identifying a database workload for which index advisor access counters are to be tracked. Each SQL statement in the database workload is executed. For each SQL statement, attribute sets are determined for which a selection predicate filters a result for an SQL statement. An output cardinality of each selection predicate is determined. A logarithmic counter for an attribute set corresponding to the selection predicate is determined based on the output cardinality of the selection predicate. The determined logarithmic counter is incremented. Respective values for logarithmic counters of the determined attributes are provided to an index advisor. The index advisor determines attribute sets for which to propose an index based on the logarithmic counters of the respective attribute sets.

Implementation of data access metrics for automated physical database design

According to some embodiments, methods and systems may be associated with a cloud computing environment. A workload prediction framework may receive observed workload information associated with a database in the cloud computing environment (e.g., a DataBase as a Service ('DBaaS')). Based on the observed workload information, a Statement Arrival Rate ('SAR') prediction may be generated. In addition, a host variable assignment prediction may be generated based on the observed workload information. The workload prediction framework may then use the SAR prediction and the host variable assignment prediction to automatically create a workload prediction for the database. A physical database design advisor (e.g., a table partitioning advisor) may receive the workload prediction and, responsive to the workload prediction, automatically generate a recommended physical layout for the database (e.g., using a cost model, the current physical layout, and an objective function).

Techniques and solutions are described for partitioning data among different types of computer-readable storage media, such as between RAM and disk-based storage. A measured workload can be used to estimate data access for one or more possible partition arrangements. The partitions arrangements can be automatically enumerated. Scores for the partition arrangements can be calculated, where a score can indicate how efficiently a partition arrangement places frequently accessed data into storage specified for frequently-accessed data and placed infrequently accessed data into storage specified for infrequently accessed data.

Workload aware data partitioning

A system includes determination of a plurality of queries of a workload, determination of a data source comprising a plurality of data rows, and determination of a sample data source based on a cardinality of each of the plurality of queries with respect to the data source and an estimated cardinality of each of the plurality of queries with respect to the data source, wherein the estimated cardinality of a query with respect to the data source is determined based on the sample data source.

Workload-based sampling

A system includes determination of a query on a table, the query including a first predicate and a second predicate, determination of a sample of the table, the sample comprising a plurality of tuples of the table, determination of a first conditional probability of selecting a tuple of the sample satisfying the first predicate and the second predicate, determination of a second conditional probability of selecting a tuple of the sample satisfying the first predicate and not satisfying the second predicate, adjustment of the first conditional probability based on the second conditional probability, a number of tuples of the sample satisfying the second predicate, a number of tuples of the sample not satisfying the second predicate, and a number of tuples of the sample satisfying the first predicate and not satisfying the second predicate, and determination of a selectivity of the query on the table based on the first adjusted conditional probability.

Selectivity estimation using non-qualifying tuples

A system for cardinality estimation for database query planning is provided. In some implementations, the system performs operations comprising determining, based on a first bounds and a second bounds for a combination of query predicates, whether a solution for the combination of the query predicates exists which includes both the first and second bounds. When the solution is determined to exist, the operations can further comprise determining at least one selectivity estimate for the combination of the query predicates. A query plan for execution of a database query comprising the query predicates can be generated based on the at least one selectivity estimate, and the database query can be executed through execution of the query plan. Related systems, methods, and articles of manufacture are also described.

Selectivity estimation for database query planning

A system for cardinality estimation for database query planning is provided. In some implementations, the system performs operations comprising selecting a subset of data from a set of data on which a database query is to be executed, the set of data including a first quantity of tuples and the subset of data including a second quantity of tuples. The operations can further comprise determining, based on evaluating one or more predicates on the subset, a third quantity of tuples in the subset which satisfy the one or more predicates. The operations can further comprise determining, based on the first quantity, the second quantity, and the third quantity, a range within the subset that comprises estimated cardinalities of the one or more predicates within a predetermined error threshold range. Related systems, methods, and articles of manufacture are also described.

Cardinality estimation for database query planning

A method, medium, and system to generate cardinality estimates for at least one query execution plan for a query by representing a given set of multidimensional data including a plurality of data points having a value by a hierarchical tree data structure including nodes, each node representing a distribution of a subset of the points in the data set and being partitioned into tiles; calculating a bounding rectangle of the data points in each node; entering all non-empty tiles into a queue; processing of the queue can continue until a computational memory space limit is reached or until a desired estimation resolution is met; and encoding the resulting tree structure.

Methods and systems for estimating the number of points in two-dimensional data

A method and system to generate cardinality estimates for at least one query execution plan for a query using a histogram, the cardinality estimation being constrained by a q-error that is a factor by which the estimate deviates, at most, from a true value of the cardinality and a threshold value which the cardinality does not exceed; use the cardinality estimation to determine an optimal query plan for executing the query; and produce an output of the optimal query plan.

Methods and systems for one dimensional heterogeneous histograms

A materialized XML view matching system and method for processing of SQLXML queries using view matching of materialized XML views. The view matching process of the embodiments of the system and method use a multi-path tree (MPT) data structure. Embodiments of the materialized XML view matching system and method construct an MPT data structure for each input query and view expression. View matching is performed on the MPT data structures to generate a set of partial matches, which then are cleaned to generate a set of candidate matches. A valid match definition is generated by testing each candidate match for different forms of compliance. Using the valid match definition, a set of valid matches is identified and extracted. For each valid match, a substitute query expression is constructed that can serve as a replacement for the original query. These substitute queries can be used to evaluate the original query.

Growing community of inventors

Mannheim, Germany

Guido Moerkotte