Red Hat, Inc.

Roland Ludwig Huss

Huamin Chen

Jinjun Xiong

Markus Thoemmes

Ricardo Noriega De Soto

One or more metrics associated with performance of a serverless function chain are received. The one or more metrics are used by a reinforcement learning (RL) model to tune the serverless function chain.

Utilizing reinforcement learning for serverless function tuning

A method includes detecting, by a webhook controller, an inference serverless function invocation. The method further includes determining that the inference serverless function can be optimized. The method further includes generating an optimized version of the inference serverless function using a graph compiler, in response to the determining. The method further includes replacing, by a processing device of the webhook controller, a storage volume in an init container of the inference serverless function with a new storage volume comprising the optimized version of the inference serverless function.

Resource constraint aware deep learning model optimization for serverless-based inference systems

A method includes monitoring, during a first time interval, traffic associated with one or more applications executed by a cluster of compute nodes and determining, in view of the traffic associated with the one or more applications during the first time interval, that the traffic is predicted to exceed a capacity threshold of the cluster of compute nodes at an end of a second time interval. The method further includes initiating startup of an additional compute node to be added to the cluster of compute nodes for executing replicas of the one or more applications.

Trajectory-based hierarchical autoscaling for serverless applications

Events to be executed by one or more serverless functions are received from an events queue. A determination that a first set of the events are to be executed by one or more serverless functions at a local host system and a second set of the events are to be executed at one or more serverless functions at a remote host system is made. The first set of the events are directed to the one or more serverless functions at the local host system and the second set of the events are directed to the one or more serverless functions at the remote host system.

Hybrid push and pull event source broker for serverless function scaling

Software performance can be automatically managed in a distributed computing environment. In one example, a system that can receive metrics information describing resource usage by a first instance of a service in a distributed computing environment. The system can also determine a quality-of-service (QoS) constraint for the service. The system can then modify a definition file based on the metrics information and the QoS constraint, the definition file being configured for deploying instances of the service in the distributed computing environment. The system can deploy a second instance of the service in the distributed computing environment using the modified definition file. As a result, the second instance can more closely satisfy the QoS constraint than the first instance.

Automatically managing performance of software in a distributed computing environment

Methods and systems for scaling computing processes within a serverless computing environment are provided. In one embodiment, a method is provided that includes receiving a request to execute a computing process in the serverless computing environment. A first node may be created within the serverless computing environment to execute the computing process. A first amount of computing resources may be assigned to the first node. It may be determined later that the first amount of computing resources are not sufficient to implement the first node. A second amount of computing resources may be determined with a vertical autoscaling process and a second node may be created within the serverless computing environment using a horizontal autoscaling process. The second node may be assigned the second amount of computing resources. The computing process may then be executed using both the first and second nodes within the serverless computing environment.

Diagonal autoscaling of serverless computing processes for reduced downtime

A method includes determining whether a code update for the service is available at a central repository of the computing environment and, in response to determining that the code update is available, retrieving the code update from the central repository. The method further includes performing a modification of the service in view of the code update.

Pull based inner-loop code deployment

Flags for a command-line interface (CLI) can be automatically determined. In one example, a system can receive a user input through the CLI to manipulate an object in a computing environment. The user input can include a flag for setting a customizable parameter of the object to a particular value. The system can also receive definition data specifying one or more customizable parameters for the object. The system can then determine one or more available flags associated with the one or more customizable parameters specified in the definition data, where the available flag(s) are usable for configuring the one or more customizable parameters of the object. Based on the available flag(s), the system can determine if the flag in the user input is valid. If so, the system can manipulate the object in the computing environment such that the manipulated object has the particular value for the customizable parameter.

Automatically determining flags for a command-line interface in a distributed computing environment

One or more metrics associated with performance of a serverless function chain are received. The one or more metrics are analyzed by a reinforcement learning (RL) model to determine a tuning factor in view of a goal provided to the RL model. The tuning factor is transmitted to a serverless function controller, wherein the serverless function controller is to utilize the tuning factor to tune the serverless function chain.

Growing community of inventors

Bavaria, Germany

Roland Ludwig Huss