Sap Se

Sybase, Inc.

Zbigniew Jerzak

Yuanzhen Ji

Thomas Heinze

Anisoara Nica

Gregor Hackenbroich

Palaniappan Gandhi

Eric Peukert

Uwe Jugel

Ibrahim Ahmed

The present disclosure involves systems, software, and computer implemented methods for providing quality-driven processing of out-of-order data streams based on a configurable quality metric. In one example, the method includes identifying a configurable quality metric defined by a user or application for executing continuous queries over a sliding window, the metric specifying a user requirement for the quality of the continuous query result. A quality threshold metric is calculated based on the configurable quality metric, the configurable quality metric associated with a size of an adaptation buffer for incoming tuples of the query. In response to receiving a signal indicating a new query result, a sliding window prior to a current time is identified. A runtime quality metric associated with the incoming tuples for the sliding window is measured, and the adaptation buffer is modified based on a difference between the quality threshold metric and the measured runtime quality metric.

Quality-driven processing of out-of-order data streams

A query rewriter associated with a database management system or visualization client rewrites a database query based on properties, characteristics, etc. of the visualization to be rendered by the visualization client. For example, the query rewriter receives an initial query and one or more visualization parameters (such as width, height and/or type of visualization) for the visualization client. The query rewriter rewrites the initial query based on the visualization parameter(s), so as to produce a data-reducing query, and then outputs the data-reducing query for execution. The query rewriter can selectively rewrite the initial query depending on size of query results of the initial query. In some example implementations, the query rewriting models a process of rasterization of geometric primitives by the visualization client, so as to facilitate error-free visualization. In many cases, the query rewriter significantly reduces the volume of query results while facilitating fast interaction with the visualization.

Pixel-aware query rewriting

The present disclosure involves systems, software, and computer implemented methods for optimizing continuous queries for hybrid execution over a stream processing engine and an in-memory database. In one example, a method may include identifying a continuous query executed at a stream processing engine, the continuous query including a plurality of operators. An optimal plan for execution of the identified continuous query at the stream processing engine is determined. For each of the plurality of operators in the determined optimal plan, an optimized placement decision for executing a particular operator in the stream processing engine or at a database system is determined. An optimized continuous query is generated from the identified continuous query based on the determined optimal placement decisions for each of the plurality of operators in the determined optimal plan. The optimized continuous query is then executed at the stream processing engine and the database system.

Optimization of continuous queries in hybrid database and stream processing systems

Various embodiments of systems and methods for a fault tolerance based query execution are described herein. Queries are received from users, the queries including operators. A multi-query optimization is performed on the operators included in the queries to obtain a query plan. A fault tolerance degree is determined for the operators included in the query plan. Based on the fault tolerance degree of the operators, nodes are assigned to the operators included in the query plan. The assigned nodes execute the operators included in the query plan to execute the queries. In one aspect, the nodes simultaneously execute the operators included in the query plan.

Fault tolerance based query execution

In an example embodiment, event stream processing is performed by first parsing an input query into a directed acyclic graph (DAG) including a plurality of operator nodes. Then a grouping of one or more of the operator nodes is created. One or more partitions are created, either by the user or automatically, in the DAG by forming one or more duplicates of the grouping. A splitter node is created in the DAG, the splitter node splits data from one or more event streams and distributes it among the grouping and the duplicates of the grouping. Then, the input query is resolved by processing data from one or more event streams using the DAG.

Event stream processing partitioning

Data is received that comprises at least one data stream derived from each of a plurality of sensors that each characterize one or more attributes of equipment components. Thereafter, using the received data and a density-based clustering algorithm that produces micro-clusters for each pair of sensors, correlated sensors having component correlations above a pre-defined threshold are identified. It can then be determined that data from two or more correlated sensors triggers at least one alert event. Subsequently, data is provided that characterizes the at least one alert event. Related apparatus, systems, techniques and articles are also described.

Self-configuring system for real-time health monitoring of manufacturing equipment

A cost monitoring system can monitor a cost of queries executing in a complex event processing system, running on top of a pay-as-you-go cloud infrastructure. Certain embodiments may employ a generic, cloud-platform independent cost model, multi-query optimization, cost calculation, and/or operator placement techniques, in order to monitor and explain query cost down to an operator level. Certain embodiments may monitor costs in near real-time, as they are created. Embodiments may function independent of an underlying complex event processing system and the underlying cloud platform. Embodiments can optimize a work plan of the cloud-based system so as to minimize cost for the end user, matching the cost model of the underlying cloud platform.

Cost monitoring and cost-driven optimization of complex event processing system

Systems and methods according to embodiments provide elasticity for complex event processing (CEP) systems. Embodiments may comprise at least the following three components: (1) incremental query optimization, (2) operator placement, and (3) cost explanation. Incremental query optimization allows avoiding simultaneous computation of identical results by performing operator-level query reuse and subsumption. Using automatic operator placement, a centralized CEP engine can be transformed into a distributed one by dynamically distributing and adjusting the execution according to unpredictable changes in data and query load. Cost explanation functionality can provide end users with near real-time insight into the monetary cost of the whole system, down to operator level granularity. Combination of these components allows a CEP system to be scaled up and down.

Growing community of inventors

Dresden, Germany

Zbigniew Jerzak