International Business Machines Corporation

Rohit Wagle

James R Giles

Louis Degenaro

Kirsten Weale Hildrum

James R Challenger

Joel Leonard Wolf

Paul Reed

Chitra Venkatramani

Xiaolan Zhang

Deepak Rajan

Vibhore Kumar

Shankar Venkataraman

Kun-Lung Wu

Eugen Schenfeld

Bugra Gedik

Henrique Andrade

Kanthi K Sarpatwar

Sunil Bharadwaj

Gabriela Jacques Da Silva

Zubair Nabi

Robert Saccone

Viswanath Nagarajan

Anju Bansal

Raghuram R Velega

Philip Shilung Yu

Wei Fan

Deepak Srinivas Turaga

Daniel Michael Kuchta

Nagui Halim

Yoonho Park

Lisa Amini

Laurent Schares

Marc A Taubenblatt

Nikhil Bansal

Michael J E Spicer

Philippe L Seto

Phillippe L Selo

Joseph R Idziorek

Methods, systems, and computer program products for cross-platform scheduling with fairness and platform-specific optimization are provided herein. A method includes determining dimensions of a set of containers in which multiple tasks associated with a request are to be executed; assigning each of the containers to a processing node on one of multiple platforms based on the dimensions of the given container, and to a platform owner selected from the multiple platforms based on a comparison of resource requirements of each of the multiple platforms and the dimensions of the given container; and generating container assignments across the set of containers by incorporating the assigned node of each container in the set of containers, the assigned platform owner of each container in the set of containers, one or more scheduling requirements of each of the platforms, one or more utilization objectives, and enforcing a sharing guarantee of each of the platforms.

Cross-platform scheduling with long-term fairness and platform-specific optimization

A distributable and serializable finite state machine and methods for using the distributable and serializable finite state machine are provided wherein finite state machine instance can be location-shifted, time-shifted or location-shift and time-shifted, for example by serializing and deserializing each instance. Each instance can be located-shifted between agents, and a persistent memory storage location is provided to facilitate both location-shifting and time-shifting. Finite state machine instances and the actions that make up each instance can be run in a distributed fashion among a plurality of agents.

Distributable serializable finite state machine

Method, system, and computer program product to enforce application dependencies, by building a data model based on inputs specifying relationships between a plurality of applications, generating, based on the data model, a list of candidate executions for a first application having a specified relationship with a second application, and outputting a visual representation of the data model, the visual representation comprising a plurality of objects representing the plurality of applications and arranged to represent the relationships between the applications, the list of candidate executions, and one or more user interface elements allowing user navigation between the first application and the second application, of the list of candidate executions.

Chaining applications

A method includes receiving, by a computing device, a plurality of data streams from plurality of sources, distributing the data streams to a plurality of sinks on multiple hosts, receiving load information indicating a load on at least one of the plurality of sinks and adjusting the distribution of the data stream accordingly and instructing the plurality of sinks to write the data streams to a distributed data store.

Dynamic streaming data dispatcher

Embodiments of the disclosure include a method for providing stream processing with runtime adaptation includes registering one or more events, wherein each of the events is associated with a stream processing application. The method also includes monitoring, by a processor, for an occurrence of the one or more events associated with the stream processing application, wherein each of the one or more events is associated with one or more runtime metrics. The method further includes receiving an event notification, wherein the event notification includes event identification and an event context and executing an adaptation of the stream processing application.

Stream processing with runtime adaptation

Embodiments of the disclosure include a system for providing stream processing with runtime adaptation, having a stream processing application that receives an incoming data stream and a runtime infrastructure configured to execute the stream processing application. The system also includes an orchestrator configured to communicate with the runtime infrastructure and the stream processing application, the orchestrator configured to perform a method. The method includes registering one or more events, wherein each of the events is associated with a stream processing application. The method also includes monitoring, by a processor, for an occurrence of the one or more events associated with the stream processing application, wherein each of the one or more events is associated with one or more runtime metrics. The method further includes receiving an event notification, wherein the event notification includes event identification and an event context and executing an adaptation of the stream processing application.

A method, apparatus, and computer program product for scheduling stream-based applications in a distributed computer system with configurable networks are provided. The method includes choosing, at a highest temporal level, jobs that will run, an optimal template alternative for the jobs that will run, network topology, and candidate processing nodes for processing elements of the optimal template alternative for each running job to maximize importance of work performed by the system. The method further includes making, at a medium temporal level, fractional allocations and re-allocations of the candidate processing elements to the processing nodes in the system to react to changing importance of the work. The method also includes revising, at a lowest temporal level, the fractional allocations and re-allocations on a continual basis to react to burstiness of the work, and to differences between projected and real progress of the work.

Method, apparatus, and computer program product for scheduling work in a stream-oriented computer system with configurable networks

Fault tolerance is provided in a distributed system. The complexity of replicas and rollback requests are avoided; instead, a local failure in a component of a distributed system is tolerated. The local failure is tolerated by storing state related to a requested operation on the component, persisting that stored state in a data store, such as a relational database, asynchronously processing the operation request, and if a failure occurs, restarting the component using the stored state from the data store.

Fault tolerance in distributed systems

Methods and systems are provided for unified job processing of interdependent heterogeneous tasks within a domain containing a plurality of nodes. Jobs containing components to be processed in the domain are submitted and are identified by type. A job control flow associated with the job type is identified and used to process the components of the job on various nodes within the domain. Multiple job control flows are handled simultaneously, and provisions are made for sharing common job components among jobs. The job control flow utilizes a finite state machine where a given instance of the finite state machine applies to a specific job type. The finite state machine can be expressed using extensible mark-up language schema.

Unified job processing of interdependent heterogeneous tasks using finite state machine job control flow based on identified job type

An illustrative embodiment of the present invention includes a method for routing at least one packet from at least one source node to at least one destination node, each node is operatively coupled to a given packet switch within a plurality of packet switches operatively coupled to a given circuit switch. The method includes a step of establishing one or more Virtual Local Area Networks (VLANs). Each VLAN comprising a subset of the plurality of packet switches operatively coupled to the given circuit switch. At least one of the one or more VLANs comprises the at least one packet switch to which the source and destination nodes are operatively coupled. The method also includes a step of routing the at least one packet from the source node to the destination node over at least one loop-free path comprised of one or more packet switches within a given VLAN.

Method and apparatus for VLAN-based selective path routing

Plug-in configurable middleware is provided for managing distributed applications. The middleware includes at least one core runtime engine configured as a plurality of concurrent instantiations on one or more hosts within a distributed architecture. These hosts can represent separate nodes or a single node within the architecture. Each core runtime engine instance provides the minimum amount of functionality required to support the instantiation of one or more plug-ins within that core runtime engine instance. Each core runtime engine instance is in communication with other concurrent core runtime engine instances and can share the functionality of plug-in instances with the other core runtime engine instances, for example through the use of proxies. A plurality of personalities representing pre-defined functions is defined and one of more of these personalities is associated with each core runtime engine instance. A plurality of pre-defined plug-ins are defined and associated with the personalities.

Distributed pluggable middleware services

Growing community of inventors

Elmsford, NY, United States of America

Rohit Wagle