International Business Machines Corporation

Jinmei Shen

Hao Wang

William Thomas Newport

Richard Alan Diedrich

A Joseph Bockhold

A method, computer program product and system for maintaining data integrity of multiple copies of a piece of data in data servers of multiple data centers, includes maintaining a cluster of catalog servers, where one of the catalog servers is selected as the master catalog server, storing the piece of data in a primary data server chosen from the data servers, duplicating the piece of data to one or more backup data servers chosen from the data servers, recording a data state information to the master catalog server, duplicating the data state information to the cluster of catalog servers, and updating the data servers and the cluster of catalog servers when the primary data server fails.

Maintaining data integrity in data servers across data centers

A method, system and computer program product manages a batch processing job by: partitioning the batch processing job for execution in a plurality of batch execution servers from a cluster of computers; designating one computer from the cluster as a primary command server that oversees and coordinates execution of the batch processing job; selecting a second computer from the cluster to serve as a failover command server; storing an object data grid structure in the primary command server; replicating the object grid structure to create and store a replicated object grid structure in the failover command server; in response to the primary command server failing, restarting, by the failover command server, execution of batch processes from the batch processing job in the plurality of batch execution servers utilizing objects within the replicated object grid structure, and executing the batch processes with processing states at the time of the failover.

Mechanism to enable and ensure failover integrity and high availability of batch processing

An apparatus, program product and method utilize on-demand propagation of routing information to reduce the overhead associated with distributing routing information throughout a distributed computing system. The staleness of the routing information utilized in routing client requests for a particular client is gauged based upon the number and/or frequency of client requests issued for that client that require rerouting, e.g., as indicated in responses returned in response to the client requests. Once the reroutes of client requests meet a desired propagation criterion, updated routing information is then requested for the client.

On-demand propagation of routing information in distributed computing system

A method, system and computer program product for managing a batch processing job is presented. The method includes partitioning a batch processing job for execution by a cluster of computers. One of the computers from the cluster of computers is designated as a primary command server that oversees and coordinates execution of the batch processing job. Stored in an object data grid structure in the primary command server is an alarm setpoint, boundaries, waiting batch processes and executing batch process states. The object data grid structure is replicated and stored as a replicated object grid structure in a failover command server. If the primary command server fails, the failover command server freezes all of the currently executing batch processes, interrogates processing states of the cluster of computers, and restarts execution of the batch processes in the cluster of computers in accordance with the processing states of the cluster of computers.

A method, apparatus, system, and signal-bearing medium that, in an embodiment, collect previous and current statistics based on previous and current access plans, respectively, and a query. The previous and current statistics are compared in response to an event, and the differences between the previous and current statistics are presented. In various embodiments, the event may be a change to logic in a database management system that performs the query, a degradation in performance of the query, or a user request. In various embodiments, the difference may be a change to the previous access plan that causes performance degradation of the query, such as missing an index. The previous access plan is copied into the current access plan if an estimated processing time for the current access plan is greater than an estimated processing time for the previous access plan. The current access plan is used to perform the query.

Maintain optimal query performance by presenting differences between access plans

An apparatus and method is described for improving access to mostly read data on network servers. The preferred embodiments more efficiently utilize replicated data servers to minimize server response time for improved performance of data access to network servers by workload managing client requests across the primary server and all replicated servers when it is possible to do so. In preferred embodiments, a load balancer supplies the most current data for mostly read data transactions while maximizing server usage by workload managing client requests across the primary server and all replicated servers. Client requests are managed by a load balancer in the workload manager. Client requests are sent by the load balancer to replicated servers when a routing table (stale data marker list) indicates that the data is in a safe period. Clients are directed exclusively to the primary server only during data update times.

Efficient handling of mostly read data in a computer server

In an embodiment, data is partitioned into partitions, which are divided into levels. The levels are ordered by creation times of the levels. A request is received at a current partition, which includes a key that identifies a field in a record and a value for the key. A determination is made whether the value exists in the field in the current partition. If the determination is false, a message is sent from the current partition to a next-older partition, and the message instructs the next-older partition to move the record with the value from the next-older partition to the current partition. If the determination is true, the record with the value in the field is moved from the current partition to a next-newer partition if the next-newer partition sent the request, and the record is deleted from the current partition.

Moving records between partitions

A method, apparatus, and program product that facilitates the implementation of grid or distributed network computing by automatically organizing a group of computers into a hierarchical, or tree networked system. The computer servers are selected into subgroups with each subgroup selecting a server to act as its leader. The leader is responsible for reporting performance characteristics of the servers in the subgroup to a master catalog server housed in a centralized management facility. The hierarchical system contemplated by embodiments of the invention reduces the number of messages that must be sent between the servers in the system necessary to monitor and manage the system. This reduction in the number of messages that must be sent between the servers in the system reduces the amount of server resources dedicated to overhead monitoring and managing, freeing these resources to be dedicated to performing the common processing task that grid or distributed network system was established to perform.

Grid computing implementation

A method, computer program product or computer system for selecting a target data server from a plurality of data servers in an information system to service a request of a client with a data quality requirement, includes determining a formalized data quality of the client using the data quality requirement, using the formalized data quality of the client and finding a cluster of data servers from the plurality of data servers that can meet the data quality requirement of the client, choosing a select data server from the cluster of data servers, verifying if the select data server can still meet the data quality requirement of the client, and if so, selecting the select data server as the target data server for processing the request, and if not, selecting another data server as the target data server for processing the request.

Asymmetric dynamic server clustering with inter-cluster workload balancing

A method for handling service requests in a data processing system is disclosed. In response to a service request, a priority of the service request is determined by mapping a context of the service request to a service level agreement (SLA). SLA-aware and timeout-conscious and preemptive and replayable lock manager is designed to account for different operational goals and different importance to satisfy various SLAs. An average processing time of the service request is then calculated based on statistical results. The service request will be allowed to process if the determined average processing time meets historical guarantee situations.

Method and apparatus for handling service requests in a data processing system

In an embodiment, in response to a request, a connection is selected from a pool of connections to a server. If the connection is stale and the number of stale connections encountered is greater than an adaptive threshold, then recovery is performed on all of the connections in the pool. If the number of stale connections is not greater than the adaptive threshold, then recovery is performed on the stale connection. A decision is made whether the connection is stale by sending the request to the server via the connection and detecting whether the sending encountered an error and by receiving a response from the server via the current connection and detecting whether the response indicates that the request encountered an error at the server.

Recovering a pool of connections

A method, system and computer program product for dynamically modifying target server specifications to improve the success rate of client requests in a data network. A client's initial request is transmitted by a client router to a dynamically updating target server, based on information in a client's routing table. A Dynamic Server Specifications (DSS) utility automatically revises one or more target specifications of a client's request that is not initially fulfilled. The DSS utility then initiates the transmission of the modified request to an alternate server. If the request to the alternate server is also not fulfilled, the DSS utility may further modify the request to target an arbitrary server in order to download a current routing table. When the client receives a current routing table, the initial request is revised based on the current routing information, with full constraints re-established, and forwarded to the relevant target server.

Automatic relaxing and revising of target server specifications for enhanced requests servicing

A method and system for processing a transaction request in a distributed data handling architecture. Responsive to receiving a transaction request at a master transaction server, a transaction identifier that identifies a data operation specified by the received transaction request is recorded within a local memory of a replica server. Data operation results are committed master transaction server memory and responsive to committing the data within the master transaction server memory, a post commit signal is sent to the replica server. Data operation results are also committed within local memory of the replica server, and responsive to a failover condition that prevents the master transaction server from sending the post commit signal, the recorded transaction identifier is utilized to synchronize data stored locally within the replica server with data stored within a persistent backend data store shared by the master transaction server and the replica server.

Failover processing in multi-tier distributed data-handling systems

A method, apparatus, system, and signal-bearing medium that, in an embodiment, accumulate multiple work units into a bundle and send the bundle to a selected active server. The active server is selected based on the relative costs for the work units and costs for servers. In an embodiment, selecting the active server includes determining relative work unit costs and relative server costs, calculating total weighted costs for the servers based on the relative work unit costs and the relative server costs, and selecting the active server that has the lowest total weighted cost. The active server may either process a work in the bundle locally or send the work unit to a passive server for processing. The active server determines whether each work unit can be executed at the active server. If the work unit can be executed on the active server, the active server executes the work unit and creates local results. If the work unit cannot be executed on the server, the active server sends the work unit to a passive server, and the passive server creates remote results. The active server receives the remote results from the passive server, combines the local results with the remote results into combined results, and sends the combined results to a client. In various embodiments, branch and dependency data are included in the bundle, which the active server uses to order and conditionally process for the work units.

Bundling and sending work units to a server based on a weighted cost

Discrepancies between client replication data quality information and server replication data quality information may be addressed by recording and transmitting the transaction processing rate for the server. The transaction processing rate may be used in conjunction with the number of transactions the server is behind to determine the transaction delay. The transaction delay may also be used in conjunction with the transaction processing rate to calculate the transaction lag. Using this information, the client and server may have a similar understanding of the replication data quality. This information may be cached and used to determine what servers requests should be routed to. Each time a client makes a request, the transaction processing rate and/or the transaction lag for the given server may be inserted into the replication data stream. The values inserted into the replication data stream may invalidate and replace the cached values.

Efficient approach with the toleration of stale data to dynamically transform and unify data quality in client and server with continuous transaction flows

Growing community of inventors

Rochester, MN, United States of America

Jinmei Shen