International Business Machines Corporation

Vibhore Kumar

Kun-Lung Wu

Bugra Gedik

Henrique Andrade

Deepak Rajan

Rohit Madhukar Khandekar

Joel Leonard Wolf

Martin J Hirzel

Kirsten Weale Hildrum

Rohit Wagle

Sujay Sunil Parekh

Robert J Soule

Gabriela Jacques Da Silva

Giuliano Losa

Deepak Srinivas Turaga

Nagui Halim

Eric P Bouillet

Srinivasan Parthasarathy

Anand Ranganathan

Chitra Venkatramani

Laurent Sebastien Mignet

Leman Akoglu

Robert Grimm

Kuldeep K Nigam

Kung-Lung Wu

Swati Shah

Sai Wu

Jim Challenger

Joseph R Idziorek

A method includes receiving, by a computing device, a plurality of data streams from plurality of sources, distributing the data streams to a plurality of sinks on multiple hosts, receiving load information indicating a load on at least one of the plurality of sinks and adjusting the distribution of the data stream accordingly and instructing the plurality of sinks to write the data streams to a distributed data store.

Dynamic streaming data dispatcher

A method of enabling compiler assisted parallelization of one or more stream processing operators in a stream processing application, which consists of a data flow graph with operators as vertices connected by streams. The method includes specifying a parallelized version of one or more of the operators, with a parameterized degree of parallelism, in the stream application, evaluating whether or not to use the parallelized operator, deciding the degree of parallelism of the parallelized operator, if there is a need for a parallelized operator.

System and method for compiler assisted parallelization of a stream processing operator

A method for allocating parallel, independent, data tasks includes receiving data tasks, each of the data tasks having a penalty function, determining a generic ordering of the data tasks according to the penalty functions, wherein the generic ordering includes solving an aggregate objective function of the penalty functions, the method further including determining a schedule of the data tasks given the generic ordering, which packs the data tasks to be performed.

Scheduling parallel data tasks

Data analysis applications include model building components and stream processing components. To increase utility of the data analysis application, in one embodiment, the model building component of the data analysis application is managed. Management includes resource allocation and/or configuration adaptation of the model building component, as examples.

Managing resource allocation or configuration parameters of a model building component to build analytic models to increase the utility of data analysis applications

Embodiments of the disclosure include a method for providing stream processing with runtime adaptation includes registering one or more events, wherein each of the events is associated with a stream processing application. The method also includes monitoring, by a processor, for an occurrence of the one or more events associated with the stream processing application, wherein each of the one or more events is associated with one or more runtime metrics. The method further includes receiving an event notification, wherein the event notification includes event identification and an event context and executing an adaptation of the stream processing application.

Stream processing with runtime adaptation

Embodiments of the disclosure include a system for providing stream processing with runtime adaptation, having a stream processing application that receives an incoming data stream and a runtime infrastructure configured to execute the stream processing application. The system also includes an orchestrator configured to communicate with the runtime infrastructure and the stream processing application, the orchestrator configured to perform a method. The method includes registering one or more events, wherein each of the events is associated with a stream processing application. The method also includes monitoring, by a processor, for an occurrence of the one or more events associated with the stream processing application, wherein each of the one or more events is associated with one or more runtime metrics. The method further includes receiving an event notification, wherein the event notification includes event identification and an event context and executing an adaptation of the stream processing application.

A method, an apparatus and an article of manufacture for processing a random-walk based vertex-proximity query on a graph. The method includes computing at least one vertex cluster and corresponding meta-information from a graph, dynamically updating the clustering and corresponding meta-information upon modification of the graph, and identifying a vertex cluster relevant to at least one query vertex and aggregating corresponding meta-information of the cluster to process the query.

Vertex-proximity query processing

Managing resource allocation and configuration of model building components of data analysis applications

A method, system and computer program product for matching an entry of a list to data. The method includes the steps of: generating source code which matches at least a part of said list with said data; compiling said source code into binary code; and determining a match between said list and said data using said compiled binary code wherein at least one of the steps is carried out using a computer device.

Matching an entry of a list to data

A method for scheduling a data processing job includes receiving the data processing job formed of a plurality of computing units, combining the plurality of computing units into a plurality of sets of tasks, each set including tasks of about equal estimated size, and different sets having different sized tasks, and assigning the tasks to a plurality of processors using a dynamic longest processing time (DLPT) scheme.

Reducing the response time of flexible highly data parallel task by assigning task sets using dynamic combined longest processing time scheme

Data sharing is facilitated in stream processing environments, including distributed stream processing environments. A processor of the stream processing environment obtains at least one of usage information for shared data of the stream processing environment, one or more pre-declared characteristics of the shared data, or performance information relating to the stream processing environment. Based on at least one of the usage information, the one or more pre-declared characteristics or the performance information, code is generated for managing the shared data.

Data sharing in a stream processing system

A virtual execution environment (VEE) for a streaming Intermediate Language (IL), wherein the streaming IL represents a streaming program, communicates streaming data in queues, stores data-at-rest in variables, and determines data by functions, where inputs are read from the queues and the variables, and outputs are written to the queues and the variables.

Virtual execution environment for streaming languages

State sharing is facilitated in stream processing environments, including distributed stream processing environments. A customized shared state implementation representing the state to be shared is automatically created based on at least one of user preferences, hints of usage, and system performance.

Growing community of inventors

Hawthorne, NY, United States of America

Vibhore Kumar