International Business Machines Corporation

Wenzhi Cui

Yu Gu

Kanak Behari Agarwal

Eric Rozner

Wesley Michael Felter

A type of a request that is currently being processed at a system is determined. A distribution is selected from a set of processing time distributions, the distribution forming a model that is applicable to the type. A threshold point is computed for the model. A processing time that exceeds a threshold point processing time is regarded as exhibiting tail latency. Tail latency includes a delay in processing of the request due to a reason other than a utilization of a resource of the system exceeding a threshold utilization and a size of a queue in the system exceeding a threshold size. An evaluation is made that the request will experience tail latency during processing at the system and the processing of the request at the system is aborted. The request is offloaded for processing at a peer system in a load-balanced group of systems.

Tail latency-based job offloading in load-balanced groups

A determination is made at an application executing in conjunction with a switch that a request packet sent to a virtualized computing entity (VCE) connected to the switch should be offloaded to a second VCE. A destination of the request packet is changed from the VCE to the second VCE, to form a first modified request packet. An evaluation is made whether a response packet from the second VCE, which is responsive to the request packet, is usable in the data communication with a sender of the request packet. When the evaluating is affirmative, the first modified request packet is caused to be transmitted from the switch to the second VCE.

Growing community of inventors

Austin, TX, United States of America

Wenzhi Cui