Teradata Us, Inc.

Xin Zhou

Olli Pekka Kostamaa

Yu Xu

A system, method, and computer-readable medium that facilitate dynamic skew avoidance are provided. The disclosed mechanisms advantageously do not require any statistic information regarding which values are skewed in a column on which a query is applied. Query selectivity is evaluated at a check point and thereby facilitates accurate detection of an overloaded processing module. The successful detection of an overloaded processing module causes other processing modules to stop sending more skewed rows to the overloaded processing module. Detection of an overloaded processing module is made when the overloaded processing module has received more rows than a target number of rows. Further, skewed rows that are maintained locally rather than redistributed to a detected processing module may result in more processing modules becoming overloaded. Advantageously, the disclosed mechanisms provide for a final redistribution adjustment to provide for even distribution of rows among all processing modules.

System, method and computer-readable medium for dynamic skew avoidance for generic queries

A system, method, and computer-readable medium for dynamic detection and management of data skew in parallel join operations are provided. Rows allocated to processing modules involved in a join operation are redistributed among the processing modules by a hash redistribution of the join attributes. Receipt by a processing module of an excessive number of redistributed rows having a skewed value on the join attribute is detected by a processing module which notifies other processing modules of the skewed value. Processing modules then terminate redistribution of rows having a join attribute value matching the skewed value and either store such rows locally or duplicate the rows. The processing module that has received an excessive number of redistributed rows removes rows having a skewed value of the join attribute from a redistribution spool allocated thereto and duplicates the rows to each of the processing modules. The join operation is completed by performing a local join at each processing module and merging the results of the local join operations.

System, method, and computer-readable medium for dynamic detection and management of data skew in parallel join operations

A method executed by a plurality of processors in a database system to implement temporal aggregate queries. The database system has a temporal table. A plurality of consecutive temporal partitions are defined and allocated to the respective processors. A query is received to aggregate at least one attribute of the temporal table. In response to the query, a plurality of timestamps are identified in the temporal table, the respective timestamps are allocated to the respective temporal partitions, the value changes in the at least one attribute at the plural timestamps are computed within each partition using the respective processors to which each partition is allocated, and the value changes within each partition are aggregated using the respective processors to which each partition is allocated to produce an aggregate value for the query.

Parallel algorithm to implement temporal aggregate queries

A system, method, and computer-readable medium for optimizing execution of a join operation in a parallel processing system are provided. A plurality of processing nodes that have at least one row of one or more tables involved in a join operation are identified. For each of the processing nodes, respective counts of rows that would be redistributed to each of the processing nodes based on join attributes of the rows are determined. A redistribution matrix is calculated from the counts of rows of each of the processing nodes. An optimized redistribution matrix is generated from the redistribution matrix, wherein the optimized redistribution matrix provides a minimization of rows to be redistributed among the nodes to execute the join operation.

System, method, and computer-readable medium for reducing row redistribution costs for parallel join operations

A system, method, and computer-readable medium for optimizing join operations in a parallel processing system are provided. A respective set of rows of a first table and a second table involved in a join operation are distributed to each of a plurality of processing modules. The join operation comprises a join on a first column of the first table and a second column of the second table. Each of the plurality of processing modules redistributes at least a portion of the rows of the first table distributed thereto substantially equally among the other processing modules and duplicates at least a portion of the rows of the second table distributed thereto among the plurality of processing modules. The disclosed optimization mechanisms provide for reduced spool space requirements for execution of the parallel join operation.

System, method, and computer-readable medium for duplication optimization for parallel join operations on similarly large skewed tables

A query is received to perform a temporal aggregate on at least one attribute of a temporal table. In response to the query, value changes of the at least one attribute at timestamps are computed, and the value changes are aggregated to produce an aggregate value for the query.

Growing community of inventors

Monterey Park, CA, United States of America

Xin Zhou