International Business Machines Corporation

Jian Tan

Li Feng Zhang

Xiaoqiao Meng

Shicong Meng

Yandong Wang

Zhenhua Hu

Alicia Elena Chin

Min Li

Michael Feiman

Parijat Dube

Liangzhao Zeng

Yonggang Hu

Nicholas C Fuller

Zhimin Lin

Michel Hack

Ting Wang

Kang-Won Lee

Michel H T Hack

Ting He

Yang Lei

Yuting Ji

Xiao Yu

Valentina Salapura

Minkyong Kim

Swapna Buccapatnam Tirumala

Minghong Lin

Various embodiments manage dynamic memory allocation data. In one embodiment, a set of memory allocation metadata is extracted from a memory heap space. Process dependent information and process independent information is identified from the set of memory allocation metadata based on the set of memory allocation metadata being extracted. The process dependent information and the process independent information at least identify a set of virtual memory addresses available in the memory heap space and a set of virtual memory addresses allocated to a process associated with the memory heap space. A set of allocation data associated with the memory heap space is stored in a persistent storage based on the process dependent information and the process independent information having been identified. The set of allocation data includes the process independent allocation information and a starting address associated with the memory heap space.

Restorable memory allocator

An embodiment of the invention includes a method for executing commands in a distributed computing environment. The method receives a plurality of distributed commands from one or more devices. The method determines a global command execution order for executing the received plurality of distributed commands. The method dispatches the received plurality of distributed commands to a plurality of servers hosting a plurality of corresponding shards in the distributed computing environment, where a given distributed command corresponds to one or more of a given shard if the given distributed command pertains to a state hosted by the one or more given shard. The method executes, by the one or more given shard, the given distributed command, where the execution is deterministic, and where a result of deterministic execution of the given distributed command is unanimous among the one or more given shard.

Command processing in distributed computing systems

Implementing a fair share of resources among one or more scheduling peers. Resource allocations are received for a plurality of scheduling peers. For each scheduling peer, a usage percentage difference is determined between their respective usage percentage and configured share ratio. For a first competing peer that is served more than a second competing peer, resource allocation is adjusted such that resources from the first competing peer are allocated to the second competing peer based, at least in part, on a time decay factor function that gives less weight to the usage percentage difference as an age of the usage percentage difference increases.

Hierarchical fairshare of multi-dimensional resources

A request arrival rate is obtained at a given computing node in a computing network comprising a plurality of distributed computing nodes. A topology of the computing network is determined at the given computing node so as to identify neighboring computing nodes with respect to the given computing node. A probability is computed at the given computing node based on the obtained request arrival rate and the detected network topology. The computed probability is used to select a decision from a set of decision candidates in response to a request received at the given computing node in a given time slot. The selected decision is a decision with a top average reward attributed thereto across the given computing node and the neighboring computing nodes determined based on information shared by the neighboring computing node with the given computing node.

Decision processing and information sharing in distributed computing environment

A method to share remote DMA (RDMA) pointers to a key-value store among a plurality of clients. The method allocates a shared memory and accesses the key-value store with a key from a client and receives an information from the key-value store. The method further generates a RDMA pointer from the information, maps the key to a location in the shared memory, and generates a RDMA pointer record at the location. The method further stores the RDMA pointer and the key in the RDMA pointer record and shares the RDMA pointer record among the plurality of clients.

Efficient remote pointer sharing for enhanced access to key-value stores

A method to determine a hierarchical fair share of resources among one or more scheduling peers. The method determines a greatest fairness difference between the usage percentage difference of a most-served competing peer and that of a least-served competing peer. The method determines a greatest accumulated fairness difference between the accumulated usage percentage difference of the most-served competing peer and that of the least-served competing peer. The method adjusts a resource allocation such that resources from the most-served competing peer are allocated to the least-served competing peer if an adjustment condition is met, wherein the adjustment condition is based on one or both of i) the greatest fairness difference, and ii) whether the greatest accumulated fairness difference exceeds at least one threshold.

Distributed game commands in massive multiplayer online games. A game engine receives commands and determines the order of their execution based on a global order policy using sequencers. The sequencers dispatch the commands to appropriate game engine instances on shards hosting game states. Engine instances execute the commands according to the global order established by the sequencers.

Embodiments relate to methods, systems and computer program products for cache management in a Remote Direct Memory Access (RDMA) data store. Aspects include receiving a request from a remote computer to access a data item stored in the RDMA data store and creating a lease including a local expiration time for the data item. Aspects further include creating a remote pointer to the data item, wherein the remote pointer includes a remote expiration time and transmitting the remote pointer to the remote computer, wherein the lease is an agreement that that the remote computer can perform RDMA reads on the data item until the remote expiration time.

Cache management for RDMA data stores

There is provided a method, a system and a computer program product for improving performance and fairness in sharing a cluster of dynamically available computing resources among multiple jobs. The system collects at least one parameter associated with availability of a plurality of computing resources. The system calculates, based on the collected parameter, an effective processing time each computing resource can provide to each job. The system allocates, based on the calculated effective processing time, the computing resources to the multiple jobs, whereby the multiple jobs are completed at a same time or an approximate time.

Scheduling mapreduce jobs in a cluster of dynamically available servers

A method, apparatus, and computer program product for configuring a computer cluster. Job information identifying a data processing job to be performed is received by a processor unit. The data processing job to be performed comprises a plurality of stages. Cluster information identifying a candidate computer cluster is also received by the processor unit. The processor unit identifies stage performance models for modeled stages that are similar to the plurality of stages. The processor unit predicts predicted stage performance times for performing the plurality of stages on the candidate computer cluster using the stage performance models and combines the predicted stage performance times for the plurality of stages to determine a predicted job performance time. The predicted job performance time may be used to configure the computer cluster.

Stage-aware performance modeling for computer cluster sizing

Methods, systems, and articles of manufacture for dynamic resource allocation in MapReduce are provided herein. A method includes partitioning input data into one or more sized items of input data associated with a MapReduce job; determining a total number of mapper components, and a total number of reducer components for the MapReduce job based on said partitioning; dynamically determining an allocation of resources to each of the total number of mapper components and reducer components during run-time of the MapReduce job, wherein said dynamically determining the allocation of resources comprises monitoring one or more utilization parameters for each of the total number of mapper components and total number of reducer components during run-time of the MapReduce job; and dynamically determining a number of concurrently executing mapper components and reducer components from the total number of mapper components and the total number of reducer components for the MapReduce job.

Growing community of inventors

Wappingers Falls, NY, United States of America

Jian Tan