Systems, methods and computer program products for improving placement performance of message transforms by exploiting guided replication

Abstract

Systems, methods and computer program products for optimizing an objective function (e.g. the overall end-to-end runtime latency) of a flow graph of message transformations which are placed onto an overlay network of broker machines by replicating stateless transformations in a guided manner. Exemplary embodiments include a guided replication method, including given an initial transformation graph and a network topology, running a placement algorithm to obtain top N best task-to-broker assignments, identifying the R-task which forms the performance bottleneck, checking if the objective function improvement drops below a designated threshold, if yes, then selecting the optimal task-to-broker assignment and the program terminates; otherwise, replicating the R-task, optimally distributing the data load among R-task's replicas, deriving a set of starting points from the above obtained top N task-to-broker assignments, and re-executing the placement algorithm with these starting points in an iterative fashion.