International Business Machines Corporation

Cisco Technology, Inc.

Hao Wang

Jinmei Shen

Yunyao Li

Shivakumar Vaithyanathan

Rajasekar Krishnamurthy

Prithviraj Sen

Sang Don Han

William Thomas Newport

Shalesh Thusoo

Richard Alan Diedrich

Charles Akum Njinda

A Joseph Bockhold

One embodiment provides for a method for evaluation of an artificial intelligence (AI) service, the method includes partitioning, by a processor, data into in-domain data and out-of-domain data. The processor defines held-out data from both of the in-domain data and the out-of-domain data for evaluation by each of domain and sub-domain based on building a taxonomy of both domains and sub-domains for the AI service. The held-out data is excluded from training data used for training the AI service. The processor further determines distribution underlying performance metrics for the held-out data using bootstrap validation processing. The processor also determines performance guarantees for multiple settings conditioned on multiple characteristics of an application scenario for the held-out data of the taxonomy based on the underlying performance metrics.

Workflow for evaluating quality of artificial intelligence (AI) services using held-out data

One embodiment provides for a method for evaluation of an artificial intelligence (AI) service, the method includes partitioning, by a processor, data into in-domain data and out-of-domain data. The processor defines held-out data from the in-domain data and the out-of-domain data for evaluation by domain and sub-domain based on building a taxonomy of domains and sub-domains for the AI service. The processor further determines distribution underlying performance metrics for the held-out data using statistical processing. The processor also determines performance guarantees for multiple settings conditioned on multiple characteristics of an application scenario for the held-out data of the taxonomy based on the underlying performance metrics. The processor further provides confidence intervals based on the performance guarantees.

A method, computer program product and system for maintaining data integrity of multiple copies of a piece of data in data servers of multiple data centers, includes maintaining a cluster of catalog servers, where one of the catalog servers is selected as the master catalog server, storing the piece of data in a primary data server chosen from the data servers, duplicating the piece of data to one or more backup data servers chosen from the data servers, recording a data state information to the master catalog server, duplicating the data state information to the cluster of catalog servers, and updating the data servers and the cluster of catalog servers when the primary data server fails.

Maintaining data integrity in data servers across data centers

A computer-implemented method, computer program product, and system are provided for implementing a cache offloader. A current cache memory usage is compared with a memory threshold. Responsive to the current cache memory usage exceeding the memory threshold, cache records are selected to be offloaded. Available memory in a plurality of computer systems is identified and the selected cache records are sent to the identified available memory. Transactional systems are dynamically enabled to use memory cache across multiple connected computer systems on demand eliminating conventional evictor and data miss issues that adversely impact performance.

Implementing cache offloading

A method, system and computer program product manages a batch processing job by: partitioning the batch processing job for execution in a plurality of batch execution servers from a cluster of computers; designating one computer from the cluster as a primary command server that oversees and coordinates execution of the batch processing job; selecting a second computer from the cluster to serve as a failover command server; storing an object data grid structure in the primary command server; replicating the object grid structure to create and store a replicated object grid structure in the failover command server; in response to the primary command server failing, restarting, by the failover command server, execution of batch processes from the batch processing job in the plurality of batch execution servers utilizing objects within the replicated object grid structure, and executing the batch processes with processing states at the time of the failover.

Mechanism to enable and ensure failover integrity and high availability of batch processing

In an embodiment, a plurality of keys are evicted from a first cache at a primary server. A first key is evicted from the first cache because the first key satisfied a first evict policy during a first time range. A second key is evicted from the first cache because the second key satisfied the first evict policy during a second time range, but the second key did not satisfy the first evict policy during the first time range. The plurality of keys are added to a batch file. If the batch file is full, the batch file is sent to a replica server, and the batch file requests the replica server to evict the plurality of keys from a second cache at the replica server. If the batch file is not full, the batch file is not sent to the replica server.

Evicting data from a cache via a batch file

An apparatus, program product and method utilize on-demand propagation of routing information to reduce the overhead associated with distributing routing information throughout a distributed computing system. The staleness of the routing information utilized in routing client requests for a particular client is gauged based upon the number and/or frequency of client requests issued for that client that require rerouting, e.g., as indicated in responses returned in response to the client requests. Once the reroutes of client requests meet a desired propagation criterion, updated routing information is then requested for the client.

On-demand propagation of routing information in distributed computing system

A method, system and computer program product for managing a batch processing job is presented. The method includes partitioning a batch processing job for execution by a cluster of computers. One of the computers from the cluster of computers is designated as a primary command server that oversees and coordinates execution of the batch processing job. Stored in an object data grid structure in the primary command server is an alarm setpoint, boundaries, waiting batch processes and executing batch process states. The object data grid structure is replicated and stored as a replicated object grid structure in a failover command server. If the primary command server fails, the failover command server freezes all of the currently executing batch processes, interrogates processing states of the cluster of computers, and restarts execution of the batch processes in the cluster of computers in accordance with the processing states of the cluster of computers.

A method, apparatus, system, and signal-bearing medium that, in an embodiment, collect previous and current statistics based on previous and current access plans, respectively, and a query. The previous and current statistics are compared in response to an event, and the differences between the previous and current statistics are presented. In various embodiments, the event may be a change to logic in a database management system that performs the query, a degradation in performance of the query, or a user request. In various embodiments, the difference may be a change to the previous access plan that causes performance degradation of the query, such as missing an index. The previous access plan is copied into the current access plan if an estimated processing time for the current access plan is greater than an estimated processing time for the previous access plan. The current access plan is used to perform the query.

Maintain optimal query performance by presenting differences between access plans

An apparatus and method is described for improving access to mostly read data on network servers. The preferred embodiments more efficiently utilize replicated data servers to minimize server response time for improved performance of data access to network servers by workload managing client requests across the primary server and all replicated servers when it is possible to do so. In preferred embodiments, a load balancer supplies the most current data for mostly read data transactions while maximizing server usage by workload managing client requests across the primary server and all replicated servers. Client requests are managed by a load balancer in the workload manager. Client requests are sent by the load balancer to replicated servers when a routing table (stale data marker list) indicates that the data is in a safe period. Clients are directed exclusively to the primary server only during data update times.

Efficient handling of mostly read data in a computer server

In an embodiment, data is partitioned into partitions, which are divided into levels. The levels are ordered by creation times of the levels. A request is received at a current partition, which includes a key that identifies a field in a record and a value for the key. A determination is made whether the value exists in the field in the current partition. If the determination is false, a message is sent from the current partition to a next-older partition, and the message instructs the next-older partition to move the record with the value from the next-older partition to the current partition. If the determination is true, the record with the value in the field is moved from the current partition to a next-newer partition if the next-newer partition sent the request, and the record is deleted from the current partition.

Moving records between partitions

A method, apparatus, and program product that facilitates the implementation of grid or distributed network computing by automatically organizing a group of computers into a hierarchical, or tree networked system. The computer servers are selected into subgroups with each subgroup selecting a server to act as its leader. The leader is responsible for reporting performance characteristics of the servers in the subgroup to a master catalog server housed in a centralized management facility. The hierarchical system contemplated by embodiments of the invention reduces the number of messages that must be sent between the servers in the system necessary to monitor and manage the system. This reduction in the number of messages that must be sent between the servers in the system reduces the amount of server resources dedicated to overhead monitoring and managing, freeing these resources to be dedicated to performing the common processing task that grid or distributed network system was established to perform.

Grid computing implementation

A method, computer program product or computer system for selecting a target data server from a plurality of data servers in an information system to service a request of a client with a data quality requirement, includes determining a formalized data quality of the client using the data quality requirement, using the formalized data quality of the client and finding a cluster of data servers from the plurality of data servers that can meet the data quality requirement of the client, choosing a select data server from the cluster of data servers, verifying if the select data server can still meet the data quality requirement of the client, and if so, selecting the select data server as the target data server for processing the request, and if not, selecting another data server as the target data server for processing the request.

Asymmetric dynamic server clustering with inter-cluster workload balancing

A method for handling service requests in a data processing system is disclosed. In response to a service request, a priority of the service request is determined by mapping a context of the service request to a service level agreement (SLA). SLA-aware and timeout-conscious and preemptive and replayable lock manager is designed to account for different operational goals and different importance to satisfy various SLAs. An average processing time of the service request is then calculated based on statistical results. The service request will be allowed to process if the determined average processing time meets historical guarantee situations.

Method and apparatus for handling service requests in a data processing system

In an embodiment, in response to a request, a connection is selected from a pool of connections to a server. If the connection is stale and the number of stale connections encountered is greater than an adaptive threshold, then recovery is performed on all of the connections in the pool. If the number of stale connections is not greater than the adaptive threshold, then recovery is performed on the stale connection. A decision is made whether the connection is stale by sending the request to the server via the connection and detecting whether the sending encountered an error and by receiving a response from the server via the current connection and detecting whether the response indicates that the request encountered an error at the server.

Growing community of inventors

Rochester, MN, United States of America

Hao Wang