Simple random sampling on pseudo-ranked hierarchical data structures in

Abstract

A random sample is obtained from an inverted tree data structure by maintaining a cardinality estimate for each intermediate node in the tree, and selecting a leaf node at random by descending from the root node and performing an acceptance/rejection selection at each intermediate node weighted proportional to the cardinality estimates of the children and weighted inversely proportional to the cardinality estimate of the intermediate node. Even though the selection of an intermediate node is based upon only an estimate of the true cardinality of the intermediate node, the error in the estimate does not cause bias in the overall sampling method because any bias in the selection of the intermediate node is cancelled by an opposite bias in the selection of the child of the intermediate node. The cardinality estimates are maintained when a leaf is inserted or deleted from the data structure by checking whether the insertion or deletion causes the cardinality estimate of a parent of the leaf node to differ from the actual cardinality by a predetermined error bound, and if so, the cardinality estimate of the parent is updated, and the checking and updating process is escalated up the tree. The error bound is adjustable for balancing rejection rate during sampling against I/O overhead of cardinality maintenance during database updates.