Emc IP Holding Company LLC

Dell Products Inc.

Emc Corporation

Landmark Graphics Corporation

Jonas Furtado Dias

Vinicius Michel Gottin

Angelo E M Ciarlini

Tiago Salviano Calmon

Adriana Bechara Prado

Vinícius Michel Gottin

Hugo De Oliveira Barbalho

Romulo D Pinho

Rômulo Teixeira De Abreu Pinho

Alex Laier Bordignon

André De Almeida Maximo

Daniel Sadoc Menasché

Edward José Pacheco Condori

Monica Barros

Owen Martin

Keshava Prasad Rangarajan

Eduardo Vera Sousa

Scott David Senften

Paulo De Figueiredo Pires

Joseph Blake Winston

Ashwani Dev

Diego Salomone Bruno

Xi Wang

Fábio André Machado Porto

Victor Bursztyn

Yania Molina Souto

David Michael Holmes

Wagner Dos Santos Vieira

Edward Pacheco

Bruno Carlos Da Cunha Costa

Chandra Yeleshwarapu

Angelo Ciarlini

Andre Maximo

Clifton Brent Walker

Roberto Nery Stelling Neto

Percy E Rivera Salas

Jaumir Valença Da Silveira Junior

Rodrigo Dias Arruda Senra

Vitor Silva Sousa

Nagaraj Srinivasan

Angelo Ernani Maia Ciarlini

André Maximo

José Carlos Costa Da Silva Pinto

Rafael Marinho Soares

Thiago De Sa Feital

Fábio A M Porto

Nagaraj Sirinivasan

Mauricio Melo Camara

A computer implemented method, computer program product, and system for managing execution of a workflow comprising a set of subworkflows, comprising optimizing the set of subworkflows using a deep neural network, wherein each subworkflow of the set of subworkflows has a set of tasks, wherein each task of the sets of tasks has a requirement of resources of a set of resources; wherein each task of the sets of tasks is enabled to be dependent on another task of the sets of tasks, training the deep neural network by: executing the set of subworkflows, collecting provenance data from the execution, and collecting monitoring data that represents the state of said set of resources, wherein the training causes the neural network to learn relationships between the states of said set of resources, the said sets of tasks, their parameters and the obtained performance, optimizing an allocation of resources of the set of resources to each task of the sets of tasks to ensure compliance with a user-defined quality metric based on the deep neural network output.

Workflow optimization

Techniques are provided for allocation of shared computing resources using source code feature extraction and machine learning techniques. An exemplary method comprises obtaining source code for execution in a shared computing environment; extracting a plurality of discriminative features from the source code; obtaining a trained machine learning model; and generating a prediction of an allocation of one or more resources of the shared computing environment needed to satisfy one or more service level agreement requirements for the source code. The generated prediction is optionally adjusted using a statistical analysis of an error curve, based on one or more error boundaries obtained by the trained machine learning model. The trained machine learning model can be trained using a set of discriminative features extracted from training source code and corresponding measurements of metrics of the service level agreement requirements obtained by executing the training source code on a plurality of the resources of the shared computing environment.

Allocation of shared computing resources using source code feature extraction and machine learning

Deep reinforcement learning techniques are provided for resource allocation in a shared computing environment. An exemplary method comprises: obtaining a specification of a workflow of a plurality of concurrent workflows in a shared computing environment, wherein the specification comprises a plurality of workflow states and one or more control variables for the workflow in the shared computing environment; evaluating values of the control variables for an execution of the concurrent workflows using a reinforcement learning agent by (i) observing the states, including a current state, and (ii) obtaining an expected utility score for combinations of the control variables for the execution of the concurrent workflows given an allocation of resources of the shared computing environment corresponding to the combination of control variables in the current state; and providing an allocation of the resources of the shared computing environment reflecting the combination having the expected utility score that satisfies a predefined score criteria.

Deep reinforcement learning for workflow optimization

Techniques are provided for adaptive look-ahead configuration for data prefetching. One method comprises, in response to a request for a data item in a storage system: obtaining a size of a look-ahead window for the request based on one of multiple available caching policies; and moving the requested data item and additional data items within the look-ahead window to the cache memory when the requested data item and/or the additional data items within the look-ahead window are not in the cache memory. The multiple available caching policies comprise a caching policy based on characteristics of an input/output workload of the storage system, or a portion thereof; and/or a caching policy based on an input/output workload of at least a portion of the storage system within a prior predefined time window. The look-ahead window size may be varied over time.

Adaptive look-ahead configuration for prefetching data in input/output operations

Techniques are provided for provenance-based software script reuse. One method comprises extracting provenance data from source code including, for example, source code fragments, wherein the extracted provenance data indicates a control flow and a data flow of the source code; encapsulating source code fragments from the source code that satisfy one or more similarity criteria as a reusable source code fragment; and providing a repository of encapsulated reusable source code fragments for reuse during a development of new software scripts. The repository of encapsulated reusable source code fragments optionally comprises a searchable database further including, for example, the provenance data, data annotations, input parameters and generated results for the corresponding source code fragment.

Provenance-based reuse of software code

Techniques are provided for allocating shared computing resources using source code feature extraction and cluster-based training of machine learning models. An exemplary method comprises: obtaining a source code corpus with source code segments for execution in a shared computing environment; extracting discriminative features from the source code segments in the source code corpus; obtaining a trained machine learning model, wherein the trained machine learning model is trained using samples of source code segments from clusters derived from clustering the source code corpus based on (i) a term frequency metric, and/or (ii) observed values of execution metrics; and generating, using the trained machine learning model, a prediction of an allocation of resources of the shared computing environment needed to satisfy service level agreement requirements for source code to be executed in the shared computing environment. The discriminative features may be extracted from the source code corpus using natural language processing techniques and/or pattern-based techniques.

Allocation of shared computing resources using source code feature extraction and clustering-based training of machine learning models

One example method includes determining a fall through threshold value for a cache, computing a length 's' of a sequence that is close to LRU eviction, and the length 's' is computed when a current fall through metric value is greater than the fall through threshold value, when the sequence length &#x2018;s&#x2019; is greater than a predetermined threshold length &#x2018;k,&#x2019; performing a first shift of an LRU position to define a protected queue of the cache, initializing a counter with a value of &#x2018;r&#x2019;, decrementing the counter each time a requested page is determined to be included in the protected queue, until &#x2018;r&#x2019;=0, and performing a second shift of the LRU position.

Sequence thrashing avoidance via fall through estimation

Reinforcement learning is used to dynamically tune cache policy parameters. The current state of a workload on a cache is provided to a reinforcement learning process. The reinforcement learning process uses the cache workload characterization to select an action to be taken to adjust a value of one of multiple parameterized cache policies used to control operation of a cache. The adjusted value is applied to the cache for an upcoming time interval. At the end of the time interval, a reward associated with the action is determined, which may be computed by comparing the cache hit rate during the interval with a baseline hit rate. The process iterates until the end of an episode, at which point the parameters of the cache control policies are reset. The episode is used to train the reinforcement learning policy so that the reinforcement learning process converges to a trained state.

Using reinforcement learning to dynamically tune cache policy parameters

Managing a cache memory in a storage system includes maintaining a queue that stores data indictive of the read requests for a particular logical storage unit of the storage system in an order that the read requests are received by the storage system, receiving a read request for a particular page of the particular logical storage unit, and removing a number of elements in the queue and resizing the queue in response to the queue being full. Managing the cache memory also includes placing data indicative of the read request in the queue, determining a prefetch metric that varies according to a number of adjacent elements in a sorted version of the queue having a difference that is less than a predetermined value and greater than zero, and prefetching a plurality of pages that come after the particular page sequentially if the prefetch metric is greater than a predefined value.

Lifetime adaptive efficient pre-fetching on a storage system

A method for optimizing performance of a storage system includes creating a structured state index from a universe of I/O traces of memory access operations in a storage system. The structured state index is validated against a target metric operational parameter of the storage system. If the structured state index has correlation against the target metric operational parameter of the storage system, the structured state index is used as input to a decision-making task. The decision-making task may be implemented as a deep neural network and the structured state index is used as input training data for the deep neural network. Once the decision-making task has been trained using the structured state index, the decision-making task is used in a predictive manner to generate a predicted target metric operational parameter of the storage system given a proposed storage policy.

Method and apparatus for optimizing performance of a storage system

The sizes of cache partitions, in a partitioned cache, are dynamically adjusted by determining, for each request, how many cache misses will occur in connection with implementing the request against the cache partition. The cache partition associated with the current request is increased in size by the number of cache misses and one or more other cache partitions is decreased in size causing cache evictions to occur from the other cache partitions rather than from the current cache partition. The other cache partitions, that are to be decreased in size, may be determined by ranking the cache partitions according to frequency of use and selecting the least frequently used cache partition to be reduced in size.

Method and apparatus for dynamically adapting sizes of cache partitions in a partitioned cache

A method of dynamically adjusting sizes of cache partitions includes, for each cache partition, estimating a number of hits that would occur on the cache partition for a set of potential size increases of the cache partition and a set of potential size decreases of the cache partition. Based on these estimates, a determination is made for each cache partition, whether to increase the size of the cache partition, maintain a current size of the cache partition, or decrease the size of the cache partition. Cache partition size increases are balanced with cache partition size decreases to allocate the entirety of the cache to the set of cache partitions without over allocating cache resources and while optimizing a sum of total cache hit rates of the set of cache partitions. A set of data structures is used to efficiently determine the estimated hit increases and decreases for each cache partition.

Method and apparatus for dynamically adapting cache size based on estimated cache performance

Managing a cache memory in a storage system includes maintaining a first queue that stores data indictive of the read requests for a particular logical storage unit of the storage system in an order that the read requests are received by the storage system and maintaining a second queue that stores data indictive of the read requests for the particular logical storage unit in a sort order corresponding to page numbers of the read requests, the second queue persisting for a plurality of iterations of read requests. A read request is received and data indicative of the read request is placed in the first queue and in the second queue while maintaining the sort order of the second queue. The second queue is used to determine a prefetch metric that varies according to a number of adjacent elements in the second queue.

Efficient pre-fetching on a storage system

Methods and apparatus are provided for process monitoring based on large-scale combinations of time series data. An exemplary method comprises generating a model from time series data for a given target time series; determining whether a first difference between measured values and predicted values based on the model exceeds a predefined threshold indicating a target prediction error; in response to a detected target prediction error, performing evaluations of (i) a neighborhood coherence comprising an average of variables of the model weighted by corresponding coefficients on a predefined neighborhood time window, and/or (ii) a second difference between a given value of at least one variable in the model and an average value of the at least one variable based on a training dataset; providing notifications when first predefined criteria based on the evaluations are satisfied; and updating the model when second predefined criteria based on the evaluations are satisfied.

Process monitoring based on large-scale combination of time series data

Growing community of inventors

Rio de Janeiro, Brazil

Jonas Furtado Dias