Microsoft Technology Licensing, LLC

Mingsheng Hong

Jonathan David Goldstein

Roger Shane Barga

Mohamed Hassan Ali

Mirek Riedewald

A new approach for handling stream imperfections based on speculative execution involves the retraction of incorrect events facilitated using operators to remove speculatively produced incorrect output. Additionally, parameters are disclosed that define a spectrum of consistency levels. A first parameter, maximum blocking time, exposes a tradeoff between a degree of speculation and latency. A second parameter, the maximum time data is remembered before being purged from the system, exposes a tradeoff between state size and correctness. Varying these two parameters produces a spectrum of consistency levels (e.g., strong, middle, weak) which address the specific tradeoffs built into other systems. Retraction is accomplished using operators that include Select, AlterLifetime, Join, Sum, Align, and Finalize.

Consistency sensitive streaming operators

A streaming operator assignment system and method for determining a streaming operator assignment that minimizes overload in a data processing system. Embodiments of the streaming operator assignment system include an optimization goals definition module, which defines optimization goals in terms of fundamental quantities that system administrators and application writers want to control, such as minimizing the worst case latency over all periods of time, or minimizing how much the system is backlogged with work. Embodiments of the streaming operator assignment system also include an optimization goals solution module that optimizes and solves a selected optimization goal. A specialized optimization technique is used to find the best operator (or load) assignment using the optimization goals to measure of the value of the assignment. This technique minimizes an optimization goal by iterating over all possible operators assignments over all possible nodes to find the operator assignment that minimizes the desired optimization goal.

Growing community of inventors

Ithaca, NY, United States of America

Mingsheng Hong