International Business Machines Corporation

Samuel H K Wong

Mi Wan Shum

Xin Yang

Xiang Sean Zhou

Brian K Caufield

DongJie Wei

Dong Jie Wei

Fan Ding

Embodiments of the present invention provide systems and methods for dynamically allocating data to multiple nodes. The method includes determining the usage of multiple buffers and the capability factors of multiple servers. Data is then allocated to multiple buffers associated with multiple active servers, based on the determined usage and capability factors, in order to keep the processing load on the multiple servers balanced.

Dynamic load balancing for data allocation to servers

A method, computer program product, and system for dynamically distributing data for parallel processing in a computing system, comprising allocating a data buffer to each of a plurality of data partitions, where each data buffer stores data to be processed by its corresponding data partition, distributing data in multiple rounds to the data buffers for processing by the data partitions, where in each round the data is distributed based on a determined data processing capacity for each data partition, and where a greater amount of data is distributed to the data partitions with higher determined processing capacities, and periodically monitoring usage of each data buffer and re-determining the determined data processing capacity of each data partition based on its corresponding data buffer usage.

Dynamic data partitioning for optimal resource utilization in a parallel data processing system

Growing community of inventors

San Jose, CA, United States of America

Samuel H K Wong