Method for sorting and distributing data among a plurality of nodes

Abstract

A method for distributing and sorting data among a plurality of nodes is described herein. After receiving a portion of a data set (e.g., a database), each node sorts its portion and estimates a partitioning of the sorted dataset among the nodes based in part on its own sorted data portion. Each node then provides a representation of its estimated partition to a master node. The master node, using the provided estimated partitions, determines a tentative partitioning and submits the tentative partitioning to each node. Each node then determines the effect the tentative partitioning using its data portion. If the effect is acceptable for each node, the tentative partitioning plan is used to partition the data. Otherwise, the tentative partitioning plan is repeatedly revised by the master node and considered by the nodes having data portions until an acceptable or optimum partitioning is determined. Each node then distributes data from its data portion that falls outside the partition assigned to the node to the appropriate node. Upon receipt of this data, each node can perform a merge sort to add the received data to the previously sorted data portion at the node.