Handling data skew for parallel execution with server mapping plans

Abstract

Skew handling techniques are provided in parallel execution for even load balancing and scaling. In a compile-time solution, a dynamic sampling query is issued to detect partition skew. The compile-time solution determines the number of skewed partitions and uses a hybrid distribution scheme where skewed partitions use a random distribution and non-skewed partitions use the original server mapping. In a runtime solution, producer server processes create partition mapping vectors that contain partition mapping information. Each producer server process sends its partition mapping vector to the query coordinator (QC). The QC receives the partition mapping vectors from the producer server processes, merges the vectors, and determines a skew result based on the merged mapping vectors and sends the skew result to the producer server processes. The producer server process can alter distribution of skewed partitions based on the skew result.