Microsoft Technology Licensing, LLC

Amazon Technologies, Inc.

Ray Manpreet Singh Matharu

Kartik Paramasivam

Yi Pan

Prateek Maheshwari

Samarth Shetty

Tate Andrew Certain

Douglas Brian Terry

Richard Shawn Bice

Derek William Richard Chen-Becker

Ramanathan Padinjarel Somanathan

Arijit Choudhury

Rashmi Krishnaiah Setty

Sanjay Shanthakumar

Abhijeet More

Sudhir Konduru

Murtaza Doctor

Abhishek Shivanna

Bharath Kumarasubramanian

Timothy Matthew Lewis

Hina Naseer

Karthikeyani Palanisami

Jagadish Venkatraman

Jacob R Maes

Asmita Barve

Shanthoosh Pazhanjur Venkataraman

Srinivasulu Punuru

Sanket Vishram Chandorkar

The disclosed embodiments provide a system for managing a stream-processing application. During operation, the system allocates a first host for an active instance of the stream-processing application that maintains a state during processing of one or more input streams. Next, the system allocates a second host for a first backup instance that recreates the state on the active instance by consuming changes to the state replicated from the active instance without consuming the input stream(s). During a failure on the first host, the system moves the active instance to the first host by stopping the first backup instance on the second host. Finally, the system launches the active instance on the second host to resume processing of the input stream(s) by the active instance using the recreated state from the first backup instance.

State replication, allocation and failover in stream processing

The disclosed embodiments provide a system for processing data. During operation, the system receives records of activity within a stream-processing system over a set of event streams, wherein each event stream in the set of event streams contains events related to a corresponding job in the stream-processing system. Next, the system indexes data in the records under a set of keys that include a first key related to jobs in the stream-processing system and a second key related to errors in the stream-processing system. The system then outputs the indexed data for use in analyzing the execution of the stream-processing system.

Stream processing diagnostics

A geographically distributed system using a hybrid, multi-master design is disclosed. The system may use global tables, for which both reads and writes may be performed locally, but with a single region serving as a master region. All strongly consistent reads and writes may be sent to the master region, whereas eventually consistent reads and writes may be performed in any region with a replica. The master region may not be statically assigned but may move from region to region. The master region may be assigned manually by client preference or may be assigned and updated automatically based on the time of day or workload characteristics.

Multi-region, multi-master replication of database tables

Growing community of inventors

Sunnyvale, CA, United States of America

Ray Manpreet Singh Matharu