Oracle International Corporation

Prabhu Thukkaram

Hoyong Park

Sandeep Bishnoi

Vitaly Bychkov

Unmesh Anil Deshmukh

Dmitrii Markovskii

Gyorgy Geiszter

Santosh Kumar

Iuliia Shmeleva

Péter Gordos

Alexandre De Castro Alves

Dmitrii Andreevich Strizhikozin

Jeffrey Toillion

Pavan L Advani

Kunal Mulay

Michal Chmielewski

Yathish Gatty

Khanderao Kand

Qing Zhong

Horst Heistermann

Albert Tam

Ilya Sergeyevich Shikalov

Devendra Jain

Ashish Mahendru

Subhrajyoti Choudhury

Natalia Nikiforova

Yogesh Bedekar

Avi Borthakur

Rajan Mahendrakumar Modi

Herbert Stiel

Simone Geib

Greg Pavlik

Kiran Prabhakar

Dimitri Laloue

Hari Gandhi

Philip Zampino

Jordan Raykov

Clemens Utschig-Utschig

Robert David Wunderlich, Jr

Munazza Bukhari

Iullia Shmeleva

A distributed event processing system is disclosed that receives a batch of events via a continuous data stream and performs the serialization of data in the batch of events. In certain embodiments, the system identifies a first data type of a first attribute for each event in a batch of events and determines a first type of data compression to be performed on data values represented by the first attribute. The system determines a first type of data compression to be performed on data values represented by the first attribute based on the first data type of the first attribute. The system then generates a first set of serialized data values for the first attribute. The system processes the first set of serialized data values against a set of one or more continuous queries to generate a first set of output events.

Data serialization in a distributed event processing system

An event processing system for processing events in an event stream is disclosed. The system can launch a first CQL engine in a cluster of CQL engines using a CQL engine tracking engine. The system can schedule, using the CQL engine tracking engine, the first CQL engine to process a batch of a continuous stream of input events related to an application. The system can track, using the CQL engine tracking engine, the first CQL engine to be scheduled for execution. The system can then execute, using the CQL engine tracking engine, the first CQL engine to process the batch of the continuous stream of input events to generate a set of output events related to the application.

Clustering event processing engines

Systems and methods for query decomposition for scalability of continuous query processing are disclosed. In some examples, it is determined whether a query is fully stateful. If it is, the query may be divided into two functions, an aggregate function and a global function. The aggregate functions may be distributed across multiple distributed nodes and executed in parallel. The global function may then receive the results from the distributed nodes and join the results to achieve the fully stateful result.

Query decomposition for scalability of continuous query processing

Techniques for processing event streams are provided that use streaming analytics to estimate a target user for dynamic assets. An exemplary technique includes receiving a batch of events that have position data for a potential target, determining one or more polygons that comprise the potential target based on the position data for the potential target, determining an approximate target viewership for the one or more polygons based on a profile or taxonomy of the potential target, associating an advertisement with the one or more polygons based on the approximate target viewership, receiving a continuous stream of events that have position data for a dynamic asset, determining that the one or more polygons include the dynamic asset based on the position data for the dynamic asset, obtaining the advertisement associated with the one or more polygons, and providing the advertisement to a sink for display on the dynamic asset.

Target user estimation for dynamic assets

Techniques for processing data streaming from one or more event sources in order to recreate a database table with database transactions for real-time analytics. An exemplary technique includes receiving a stream of events comprising change data for a table attribute, populating an in-memory table with the change data based on a token associated with an operation type and a mapping of a first attribute and a second attribute to a key attribute of the in-memory table, executing a continuous query on the change data for the key attribute of the in-memory table to generate events comprising data for a third attribute, and outputting the events having an output shape comprising the third attribute, where the events include the data for the third attribute.

Recreating an OLTP table and reapplying database transactions for real-time analytics

An event processing system for processing events in an event stream is disclosed. The system can execute instructions to receive a continuous query, apply a transformation to the continuous query to generate a query plan for the continuous query, transform the query plan using a transformation algorithm to generate a transformed query plan, receive a micro-batch stream of input events related to an application, process the input events of the micro-batch stream based at least in part on the transformed query plan to generate a set of output events related to the application, and store the set of output events related to the application in an output queue.

Complex event processing for micro-batch streaming

Systems and methods described herein are directed towards a flexible event ingestion framework. In some examples, an input source comprising information that identifies a plurality of events may be identified. Additionally, in some examples, an adapter for ingesting the information of the input source may be implemented. At least one additional component for modifying the adapter may be received. The adapter may be modified by implementing the at least one additional component with a transport component and a mapper component as part of ingesting the information. Further, a tuple for at least a first event may be generated based at least in part on the modified adapter, and the tuple may be provided to an event server.

Flexible event ingestion framework in an event processing system

An event processing system for processing events in an event stream is disclosed. The system is configured for determining a stage for a continuous query language (CQL) query being processed by an event processing system and/or determining a stage type associated with the stage. The system is also configured for determining a transformation to be computed for the stage based at least in part on the stage type and/or determining a classification for the CQL query based at least in part on a plurality of rules. The system can also be configured for generating a transformation in a Directly Acyclic Graph (DAG) of a data transformation pipeline for the stage based at least in part on the partitioning criteria for the stage. In some examples, the system can also be configured for determining a partitioning of the stage based at least in part on the transformation.

Data partitioning and parallelism in a distributed event processing system

An event processing system for distributing a continuous query, and processing events in an event stream, is disclosed. The system can synchronize timestamps across all partitions by adding a heartbeat propagator component propagating a heartbeat signal for silent partitions. The system can further include a timestamp tracker component that tracks the maximum observed timestamp across all partitions for each batch, and sends this to each partition. The system can distribute partitions of the events to the worker nodes, and instruct a heartbeat propagator component to generate a heartbeat for a first worker node corresponding to an event in a different partition. The system receives outputs from the worker nodes and combines these into a single result corresponding to a continuous query. The system can further implement a timestamp tracker to select a highest timestamp value of a previous batch, and send the value to the worker nodes.

Heartbeat propagation in a distributed stream processing system

Systems and methods for extracting tuples using dynamically generated extractor classes are disclosed. In some examples, an optimized tuple extraction class can be dynamically generated to enable more efficient tuple extraction.

Tuple extraction using dynamically generated extractor classes

A distributed event processing system is disclosed that receives continuous data streams, registers a continuous query against the data streams, and continuously executes the query as new data appears in the streams. In certain embodiments, the distributed event processing system deploys and executes applications (e.g., event processing applications) by distributing the execution of the application on a cluster of machines within the system. In certain embodiments, the system provides users with the ability to specify logical rules in an application. The system processes the logical rules by generating a DAG of transformations representing the logical rules and converts the DAG of transformations into a logical rules Resilient Distributed Dataset (RDD) DAG of transformations. The system processes events in an event batch against the RDD DAG of transformations to generate a set of output results for the user.

Integrating logic in micro batch based event processing systems

A distributed event processing system is disclosed that receives continuous data streams, registers a continuous query against the data streams, and continuously executes the query as new data appears in the streams. In certain embodiments, the distributed event processing system deploys and executes applications (e.g., event processing applications) by distributing the execution of the application on a cluster of machines within the system. In certain embodiments, the system receive updates to an application, processes the updated application against a continuous stream of data, and continuously generates updated results to a user. In certain embodiments, the system reduces delays associated with application deployment and execution when updates to an application are received. The system enables a user to view results pertaining to an updated application quickly without experiencing the typical deployment and execution delays that occur when updates to an application are made.

Framework for the deployment of event-based applications

Techniques for processing event streams are provided that retain attributes or fields from a current event or group during stream aggregation. An exemplary technique includes receiving a stream of events having an input shape having a plurality of attributes and executing a query on the stream of events. The executing includes: (i) summarizing data of a first attribute of the plurality of attributes based on an aggregate function to generate values of data for a result-set, (ii) collecting data of events with values that are identical for a second attribute of the plurality of attributes based on a group by function to group the result-set by the second attribute, and (iii) preserving data of a third attribute of the plurality of attributes based on a current function. The technique further includes outputting events having an output shape having the first attribute, the second attribute, and the third attribute.

Retaining fields from current event or group during stream aggregation

Growing community of inventors

San Ramon, CA, United States of America

Prabhu Thukkaram