Oracle International Corporation

Sandeep Bishnoi

Prabhu Thukkaram

Hoyong Park

Unmesh Anil Deshmukh

Anand Madihalli Srinivasan

Iuliia Shmeleva

Santosh Kumar

Gyorgy Geiszter

Vitaly Bychkov

Jeffrey Toillion

Pavan L Advani

Kunal Mulay

Namit Jain

Shailendra Mishra

Dmitrii Markovskii

Yathish Gatty

Péter Gordos

Devendra Jain

Mohit Thatte

Subhrajyoti Choudhury

Vikram Shukla

Alexandre De Castro Alves

Herbert Stiel

Ashish Mahendru

Yogesh Bedekar

Prathab Kali

Alexandre Alves

Iullia Shmeleva

A distributed event processing system is disclosed that receives a batch of events via a continuous data stream and performs the serialization of data in the batch of events. In certain embodiments, the system identifies a first data type of a first attribute for each event in a batch of events and determines a first type of data compression to be performed on data values represented by the first attribute. The system determines a first type of data compression to be performed on data values represented by the first attribute based on the first data type of the first attribute. The system then generates a first set of serialized data values for the first attribute. The system processes the first set of serialized data values against a set of one or more continuous queries to generate a first set of output events.

Data serialization in a distributed event processing system

An event processing system for processing events in an event stream is disclosed. The system can launch a first CQL engine in a cluster of CQL engines using a CQL engine tracking engine. The system can schedule, using the CQL engine tracking engine, the first CQL engine to process a batch of a continuous stream of input events related to an application. The system can track, using the CQL engine tracking engine, the first CQL engine to be scheduled for execution. The system can then execute, using the CQL engine tracking engine, the first CQL engine to process the batch of the continuous stream of input events to generate a set of output events related to the application.

Clustering event processing engines

Systems and methods for query decomposition for scalability of continuous query processing are disclosed. In some examples, it is determined whether a query is fully stateful. If it is, the query may be divided into two functions, an aggregate function and a global function. The aggregate functions may be distributed across multiple distributed nodes and executed in parallel. The global function may then receive the results from the distributed nodes and join the results to achieve the fully stateful result.

Query decomposition for scalability of continuous query processing

Techniques for processing event streams are provided that use streaming analytics to estimate a target user for dynamic assets. An exemplary technique includes receiving a batch of events that have position data for a potential target, determining one or more polygons that comprise the potential target based on the position data for the potential target, determining an approximate target viewership for the one or more polygons based on a profile or taxonomy of the potential target, associating an advertisement with the one or more polygons based on the approximate target viewership, receiving a continuous stream of events that have position data for a dynamic asset, determining that the one or more polygons include the dynamic asset based on the position data for the dynamic asset, obtaining the advertisement associated with the one or more polygons, and providing the advertisement to a sink for display on the dynamic asset.

Target user estimation for dynamic assets

Techniques for processing data streaming from one or more event sources in order to recreate a database table with database transactions for real-time analytics. An exemplary technique includes receiving a stream of events comprising change data for a table attribute, populating an in-memory table with the change data based on a token associated with an operation type and a mapping of a first attribute and a second attribute to a key attribute of the in-memory table, executing a continuous query on the change data for the key attribute of the in-memory table to generate events comprising data for a third attribute, and outputting the events having an output shape comprising the third attribute, where the events include the data for the third attribute.

Recreating an OLTP table and reapplying database transactions for real-time analytics

An event processing system for processing events in an event stream is disclosed. The system can execute instructions to receive a continuous query, apply a transformation to the continuous query to generate a query plan for the continuous query, transform the query plan using a transformation algorithm to generate a transformed query plan, receive a micro-batch stream of input events related to an application, process the input events of the micro-batch stream based at least in part on the transformed query plan to generate a set of output events related to the application, and store the set of output events related to the application in an output queue.

Complex event processing for micro-batch streaming

An event processing system for processing events in an event stream is disclosed. The system is configured for determining a stage for a continuous query language (CQL) query being processed by an event processing system and/or determining a stage type associated with the stage. The system is also configured for determining a transformation to be computed for the stage based at least in part on the stage type and/or determining a classification for the CQL query based at least in part on a plurality of rules. The system can also be configured for generating a transformation in a Directly Acyclic Graph (DAG) of a data transformation pipeline for the stage based at least in part on the partitioning criteria for the stage. In some examples, the system can also be configured for determining a partitioning of the stage based at least in part on the transformation.

Data partitioning and parallelism in a distributed event processing system

An event processing system for distributing a continuous query, and processing events in an event stream, is disclosed. The system can synchronize timestamps across all partitions by adding a heartbeat propagator component propagating a heartbeat signal for silent partitions. The system can further include a timestamp tracker component that tracks the maximum observed timestamp across all partitions for each batch, and sends this to each partition. The system can distribute partitions of the events to the worker nodes, and instruct a heartbeat propagator component to generate a heartbeat for a first worker node corresponding to an event in a different partition. The system receives outputs from the worker nodes and combines these into a single result corresponding to a continuous query. The system can further implement a timestamp tracker to select a highest timestamp value of a previous batch, and send the value to the worker nodes.

Heartbeat propagation in a distributed stream processing system

Techniques for processing event streams are provided that retain attributes or fields from a current event or group during stream aggregation. An exemplary technique includes receiving a stream of events having an input shape having a plurality of attributes and executing a query on the stream of events. The executing includes: (i) summarizing data of a first attribute of the plurality of attributes based on an aggregate function to generate values of data for a result-set, (ii) collecting data of events with values that are identical for a second attribute of the plurality of attributes based on a group by function to group the result-set by the second attribute, and (iii) preserving data of a third attribute of the plurality of attributes based on a current function. The technique further includes outputting events having an output shape having the first attribute, the second attribute, and the third attribute.

Retaining fields from current event or group during stream aggregation

An event processing system for processing events in an event stream is disclosed. The system is configured for configuring a stream processor to micro-batch incoming events from a stream source. The system is also configured for generating a single timestamp for a micro-batch of the incoming events and/or receiving the micro-batch of the incoming events from the stream source. The system can also be configured for assigning the single timestamp to each event of the micro-batch and/or generating separate timestamp values for each respective event of the micro-batch. In some examples, the system can also be configured for assigning, for each respective event of the micro-batch, an individual one of the separate timestamp values.

Processing timestamps and heartbeat events for automatic time progression

Techniques for implementing logical queries in a distributed stream processing system using automatic branching and joins. An exemplary technique includes determining a query is a logical query. The logical query includes two or more summaries based on different groups configured to execute in a single query stage of a stream analytics application. The technique further includes converting the logical query into one or more physical queries. The one or more physical queries are separated into individual query stages, and each of the query stages includes a summary from the two or more summaries that is based on an associated group. The technique further includes generating a directed acyclic graph for the one or more physical queries. The directed acyclic graph includes a physical query transformation for each of the individual query stages.

Growing community of inventors

Pleasanton, CA, United States of America

Sandeep Bishnoi