Transactional computation on clusters

Abstract

Computations are performed on shared datasets in a distributed computing cluster using aggressive speculation and a distributed runtime that executes code transactionally. Speculative transactions are conducted with currently available data on the assumption that no dependencies exist that will render the input data invalid. For those specific instances where this assumption is found to be incorrect—that the input data did indeed have a dependency (thereby impacting the correctness of the speculated transaction)—the speculated transaction is aborted and its results (and all transactions that relied on its results) are rolled-back accordingly for re-computation using updated input data. In operation, shared state data is read and written using only the system's data access API which ensures that computations can be rolled-back when conflicts stemming from later-determined dependencies are detected.