Microsoft Technology Licensing, LLC

Tihomir Tsvetkov Tarnavski

Olivier Nano

Gueorgui Chkodrov

Ivo José Garcia Dos Santos

Ivo Jose Garcia Dos Santos

Ramkumar Krishnan

Sebastien Peray

Marcel Tilly

Jonathan David Goldstein

Richard Douglas Hill

Craig Andrew Critchley

Krishnan Srinivasan

Eldar Akchurin

Pramod Gurunath

Mitchell G Morris

Paul F Ringseth

Roger Shane Barga

Andy Shen

Lev Novik

Bart De Smet

Savas Parastatidis

Badrish Chandramouli

Beysim Sezgin

Peter Zabback

Bart J F De Smet

Laurent Bussard

Eric Anthony Rozell

Tomer Verona

Panagiotis Periorellis

Alex Clemmer

A method is described of pre-processing real-time data to be processed by one or more real-time analytics services. The method includes: receiving, at a stateless ingress node, data from one or more client devices; transforming the data in real-time at the stateless ingress node; determining in real-time, at the stateless ingress node, a real-time analytics service for the transformed data; and forwarding in real-time the transformed data to the determined real-time analytics service for stateful processing.

Scalable data enrichment for cloud streaming analytics

Stream-based programming models allow subscriber to observe a stream of stream items received from a source, such as event notifications and updates to observed data. Presented herein is a stream-based reactive programming platform that allows observers to discover sources and streams, and to specify queries applicable to sources that results in new streams, including conditions that apply to the properties of a federation of sources. Variations of the reactive programming platform include generating a new stream from a non-reactive data source; generating a graph of sources and observers; and providing a directory of discoverable elements respectively identified by uniform resource identifiers. The reactive stream-based programming platform may also permit parameterization to configure the functionality of the elements participating in the stream; delegation of query tasks to discoverable elements of the programming platform; and federation of a heterogeneous set of elements to achieve a query using the reactive stream-based programming platform.

Stream-based reactive programming platform

Methods, systems, and computer-readable media for analyzing a query are disclosed. A particular method includes receiving data representing an event flow associated with execution of a query that includes a plurality of operators. A graph is generated that includes a plurality of nodes that corresponds to operators of the query, and edges of the graph that correspond to streams between operators of the query. The method includes receiving an input identifying a particular node of the graph, where the particular node corresponds to a particular operator of the query. In response to the input, an output associated with at least one event processed by the particular operator is displayed.

Visual analysis and debugging of event flows

Higher-order operators are subject to checkpointing as well as recovery. The state of a higher-order operator, comprising an outer subscription and one or more inner subscriptions, is captured and subsequently utilized to recover the operator. In one instance, a data representation of code that can produce an inner subscription is saved. In other instance, an outer subscription is decoupled from inner subscriptions of a higher-order operator.

Checkpointing higher order query operators

The formulation of transformations on one or more input event streams to generation one or more output event streams. Accordingly, the transformations may be considered to be a query on the original input event stream(s). The event query includes event stream source representations representing an input event stream available in a particular execution context. The event query also includes a transformation module identifying the transformation set to be performed on the input event streams in the execution context. Once the query is properly formed, an execution module may then cause the transformations to be executed upon the designated input event stream(s) to generate output event streams.

Event stream transformations

An analysis module is configured to receive data associated with an event flow. The data is received from a first analysis module (e.g., in a stack of analysis modules) or from the event flow. The analysis module is configured to execute an analysis operation on the data to generate a result. The analysis module can output the result to a second analysis module (e.g., in the stack of analysis modules) or to a user interface.

Analysis stack for an event flow

Computation can be encoded in a lightweight and optionally typed data representation. The data representation can be specified in a prefix-based notation potentially including nested function-argument pairs. Further, the data representation can comprise optional static type information associated with at least a portion of computation. Still further, the data representation can be programming language and platform independent or surfaced in a particular programming language or platform.

Lightweight optionally typed data representation of computation

A complex event processor is described which has a communications interface configured to retrieve event data by pulling it from one or more sources and to receive at least one live event stream pushed to the interface. An event processing pipeline connected to the communications interface comprises a plurality of operator nodes connected between the communications interface and a combiner node which is a node configured to combine event data from the source and from the live event stream. The communications interface is configured to retrieve events from the source and to push the events retrieved from the source along the event processing pipeline towards the combiner node. The communications interface is configured to retrieve and push the retrieved events only in response to request messages passed in an upstream direction from the combiner node to the communications interface along the pipeline.

Complex event processor for historic/live/replayed data

Methods, systems, and computer-readable media to generate a user interface (UI) to analyze a complex event processing (CEP) query are disclosed. A particular method includes receiving data representing an event flow associated with execution of a CEP query. The CEP query includes a plurality of operators. A UI including a graph is generated. The graph includes a plurality of nodes. Each node of the graph corresponds to an operator of the CEP query, and each edge of the graph corresponds to a stream between operators of the CEP query. The method includes receiving an input identifying a particular node of the graph, where the particular node corresponds to a particular operator of the CEP query. In response to the input, an operator-specific output associated with at least one event processed by the particular operator is displayed.

Visual analysis and debugging of complex event flows

Methods, systems, and computer-readable media to analyze a CEP query are disclosed. A particular analysis module is configured to receive data associated with an event flow generated by execution of a CEP query. The data is received from a first analysis module (e.g., in a stack of analysis modules) or from the event flow via a software communication interface. The analysis module is also configured to execute an analysis operation on the data to generate a result. The analysis module can output the result to a second analysis module (e.g., in the stack of analysis modules) or to a user interface via the software communication interface.

Analysis stack for complex event flows

Creating and executing a distributed stream processing operator graph based on a query. The operator graph includes movable stream algebra operators for processing events received from high volume data streams. The operators are partially compiled and distributed to computing devices for completion of the compilation and subsequent execution. During execution, the operators maintain minimal state information associated with received events via an expiration time assigned to each of the event instances. Additional events are generated and aggregated by the operators for communication to a service responsible for the query.

Growing community of inventors

Redmond, WA, United States of America

Tihomir Tsvetkov Tarnavski