Emc IP Holding Company LLC

Dell Products Inc.

Daniel Sadoc Menasché

Alex Laier Bordignon

Vinícius Michel Gottin

Vinicius Michel Gottin

Jonas Furtado Dias

Rômulo Teixeira De Abreu Pinho

Hugo De Oliveira Barbalho

Roberto Nery Stelling Neto

Tiago Salviano Calmon

Eduardo Vera Sousa

Manuel Ramón Vargas Avila

Angelo E M Ciarlini

Angelo Ernani Maia Ciarlini

Fábio A M Porto

Vinícius Michel Göttin

Deep reinforcement learning techniques and provenance-based simulation are employed for resource allocation in a shared computing environment. One method comprises: obtaining a specification of a workflow of concurrent workflows in a shared computing environment, wherein the specification comprises workflow states and one or more control variables for the workflow in the shared computing environment; obtaining a simulation model of the workflow representing different configurations of the control variables; evaluating the control variables for the concurrent workflows using a reinforcement learning (RL) agent by observing the states and obtaining an expected utility score for control variable combinations for the execution of the concurrent workflows given an allocation of resources of the shared computing environment, wherein the RL agent performs, using the simulation model, the evaluating, the obtaining the expected utility score, and/or a training of an RL model; and providing an allocation of the resources based on the expected utility score.

Deep reinforcement learning for workflow optimization using provenance-based simulation

Deep reinforcement learning techniques are provided for resource allocation in a shared computing environment. An exemplary method comprises: obtaining a specification of a workflow of a plurality of concurrent workflows in a shared computing environment, wherein the specification comprises a plurality of workflow states and one or more control variables for the workflow in the shared computing environment; evaluating values of the control variables for an execution of the concurrent workflows using a reinforcement learning agent by (i) observing the states, including a current state, and (ii) obtaining an expected utility score for combinations of the control variables for the execution of the concurrent workflows given an allocation of resources of the shared computing environment corresponding to the combination of control variables in the current state; and providing an allocation of the resources of the shared computing environment reflecting the combination having the expected utility score that satisfies a predefined score criteria.

Deep reinforcement learning for workflow optimization

Reinforcement learning is used to dynamically tune cache policy parameters. The current state of a workload on a cache is provided to a reinforcement learning process. The reinforcement learning process uses the cache workload characterization to select an action to be taken to adjust a value of one of multiple parameterized cache policies used to control operation of a cache. The adjusted value is applied to the cache for an upcoming time interval. At the end of the time interval, a reward associated with the action is determined, which may be computed by comparing the cache hit rate during the interval with a baseline hit rate. The process iterates until the end of an episode, at which point the parameters of the cache control policies are reset. The episode is used to train the reinforcement learning policy so that the reinforcement learning process converges to a trained state.

Using reinforcement learning to dynamically tune cache policy parameters

An information handling system for managing a storage system includes storage for storing profile-based cache policy performance prediction models. The information handling system also includes a storage manager that obtains an input-output profile for a workload hosted by the information handling system during a first period of time; obtains performance metrics for cache policies for the storage system using: the input-output profile, and the profile-based cache policy performance prediction models; obtains a ranking of the cache policies based on the performance metrics; selects a cache policy of the cache policies based on the rankings; and updates operation of a cache of the storage system based on the selected cache policy for a second period of time.

System and method for cache management

A method for optimizing performance of a storage system includes creating a structured state index from a universe of I/O traces of memory access operations in a storage system. The structured state index is validated against a target metric operational parameter of the storage system. If the structured state index has correlation against the target metric operational parameter of the storage system, the structured state index is used as input to a decision-making task. The decision-making task may be implemented as a deep neural network and the structured state index is used as input training data for the deep neural network. Once the decision-making task has been trained using the structured state index, the decision-making task is used in a predictive manner to generate a predicted target metric operational parameter of the storage system given a proposed storage policy.

Method and apparatus for optimizing performance of a storage system

Techniques are provided for workflow simulation using provenance data similarity and sequence alignment. An exemplary method comprises: obtaining a state of workflow executions of concurrent workflows with multiple resource allocation configurations, wherein the state comprises provenance data of the concurrent workflows; obtaining execution traces of the concurrent workflows representing different resource allocation configurations; identifying a set of states in a first execution trace and a set of states in a second execution trace as corresponding anchor states; mapping a first intermediate state to a second intermediate state between a pair of anchor states using the provenance data; generating a simulation model of the workflow executions representing the different configurations of the resource allocation; and generating new simulation traces of the workflow executions with resource allocation configurations that are not represented in the provenance data.

Workflow simulation using provenance data similarity and sequence alignment

A cache management system includes a sequentiality determination process configured to determine sequentiality profiles of a workload of IO traces as the workload dynamically changes over time. A learning process is trained to learn a correlation between workload sequentiality and cache pollution, and the trained learning process is used to predict cache pollution before the cache starts to experience symptoms of excessive pollution. The predicted pollution value is used by a cache policy adjustment process to change the prefetch policy applied to the cache, to proactively control the manner in which prefetching is used to write data to the cache. Selection of the cache policy is implemented on a per-LUN basis, so that cache performance for each LUN is individually managed by the cache management system.

Method and apparatus for adjusting cache prefetch policies based on predicted cache pollution from dynamically evolving workloads

Techniques are provided for characterizing and quantifying a sequentiality of workloads using sequentiality profiles and signatures. One exemplary method comprises obtaining telemetry data for an input/output workload; evaluating a distribution over time of sequence lengths for input/output requests in the telemetry data by the input/output workload; and generating a sequentiality profile for the input/output workload to characterize the input/output workload based at least in part on the distribution over time of the sequence lengths. Multiple sequentiality profiles for one or more input/output workloads may be clustered into a plurality of clusters. A sequentiality signature may be generated to represent one or more sequentiality profiles within a given cluster. A performance of data movement policies may be evaluated with respect to the sequentiality signature of the given cluster.

Sequentiality characterization of input/output workloads

Techniques are provided for detecting resource bottlenecks in workflow task executions using provenance data. An exemplary method comprises: obtaining a state of multiple workflow executions of multiple concurrent workflows performed with different resource allocation configurations in a shared infrastructure environment; obtaining first and second signature execution traces of a task representing first and second resource allocation configurations, respectively; identifying first and second corresponding sequences of time intervals in the first and second signature execution traces for the task, respectively, based on a similarity metric; and identifying a given time interval as a resource bottleneck of a resource that differs between the first and second resource allocation configurations based on a change in execution time for the given time interval between the first and second signature execution traces. The first signature execution trace may be obtained by disaggregating data related to batches of workflow executions.

Detection of resource bottlenecks in execution of workflow tasks using provenance data

Techniques are provided for simulation-based online workflow optimization. One method comprises obtaining a state of concurrent workflows; obtaining state similarity functions that assign a similarity score between pairs of states; generating a simulation model of the workflow executions representing different configurations of at least one control variable in the workflow executions by mapping states with a highest similarity; obtaining at least one utility function that assigns a utility score to the states in the simulation model; determining a configuration of the control variable that maximizes the utility score for states in the simulation model; and for new concurrent workflows: obtaining a current state of the new concurrent workflows; identifying a most similar state with one of the determined configurations of the control variable in the simulation model with a highest similarity to the current state; and adjusting the configuration of the control variable of the new concurrent workflows to match the corresponding configuration of the control variable of the most similar state.

Growing community of inventors

Rio de Janeiro, Brazil

Daniel Sadoc Menasché