Hewlett Packard Enterprise Development LP

Junguk Cho

Puneet Sharma

Diman Zad Tootaghaj

Dominik Stiller

Systems and methods are provided for improving autotuning procedures using stateless processing with a remote key-value store. For example, the system can implement a task launcher, a scheduler, and an agent to launch, schedule, and execute decomposed autotuning stages, respectively. The scheduling policy implemented by the scheduler may perform operations beyond a simple scheduling policy (e.g., a FIFO-based scheduling policy), which produces a high queuing delay. Compared to the traditional systems, by leveraging autotuning specific domain knowledge, queueing delay is reduced and resource utilization is improved.

Context-aware and stateless deep learning autotuning framework

Example implementations relate to scheduling of jobs for a plurality of graphics processing units (GPUs) providing concurrent processing by a plurality of virtual GPUs. According to an example, a computing system including one or more GPUs receives a request to schedule a new job to be executed by the computing system. The new job is allocated to one or more vGPUs. Allocations of existing jobs are updated to one or more vGPUs. Operational cost of operating the one or more GPUs and migration cost of allocating the new job are minimized and allocations of the existing jobs on the one or more vGPUs is updated. The new job and the existing jobs are processed by the one or more GPUs in the computing system.

Scheduling jobs on graphical processing units

Systems and methods are provided for improving autotuning procedures. For example, the system can implement a task launcher, a scheduler, and an agent to launch, schedule, and execute decomposed autotuning stages, respectively. The scheduling policy implemented by the scheduler may perform operations beyond a simple scheduling policy (e.g., a FIFO-based scheduling policy), which produces a high queuing delay. By leveraging autotuning specific domain knowledge, this may help reduce queuing delay and improve resource utilization that is otherwise found in traditional systems.

Deep learning autotuning task optimization

Example implementations relate to a proactive auto-scaling approach. According to an example, a machine-learning prediction model is trained to forecast future serverless workloads during a window of time for an application running in a public cloud based on past serverless workload information associated with the application by performing a training process. During the window of time, serverless workload information associated with the application is monitored. A future serverless workload is predicted for the application at a future time within the window, based on the machine learning prediction model. Prior to the future time, containers within the public cloud executing the application are pre-warmed to accommodate the predicted future serverless workload by issuing fake requests to the application to trigger auto-scaling functionality implemented by the public cloud.

Proactively accomodating predicted future serverless workloads using a machine learning prediction model

Example implementations relate to a proactive auto-scaling approach. According to an example, a target performance metric for an application running in a serverless framework of a private cloud is received. A machine-learning prediction model is trained to forecast future serverless workloads during a window of time for the application based on historical serverless workload information. The serverless framework is monitored to obtain serverless workload observations for the application. A future serverless workload for the application at a future time is predicted by the trained machine learning prediction model based on workload observations. A feedback control system is then used to output a new number of replicas based on a current value of the performance metric, the target performance metric and the predicted future serverless workload. Finally, the serverless framework is caused to scale and pre-warm a number of replicas supporting the application to the new number.

Growing community of inventors

San Jose, CA, United States of America

Junguk Cho