Cross-stream transactions in a streaming data storage system

Abstract

The described technology is generally directed towards cross-stream transactions in a streaming data storage system, which allows a writer application to commit multiple events to distinct data streams in a single transaction. The system creates a cross-stream transaction for a writer application, and the writer application adds events to the cross-stream transaction, indicating which destination data stream(s) each event's data is to be appended. The system adds the event to a subordinate transaction created for each specified data stream. Upon committing the cross-stream transaction, the system coordinates the committing of the subordinate transactions to their respective data streams. Alternatively, an application writer can request that an event be committed to multiple writer-specified data streams via a cross-stream micro-transaction, in which the streaming data storage system obtains the appropriate segment store instances and segments for each data stream, and coordinates the direct appending of the event to the appropriate segments.