Emc IP Holding Company LLC

Dell Products Inc.

Emc Corporation

Alex Laier Bordignon

Rômulo Teixeira De Abreu Pinho

Angelo E M Ciarlini

Daniel Sadoc Menasché

Vinicius Michel Gottin

Vinícius Michel Gottin

Jonas Furtado Dias

Edward José Pacheco Condori

Hugo De Oliveira Barbalho

Paulo De Figueiredo Pires

Roberto Nery Stelling Neto

Marcello Luiz Rodrigues De Campos

Tiago Salviano Calmon

Eduardo Vera Sousa

Manuel Ramón Vargas Avila

André De Almeida Maximo

Wagner Dos Santos Vieira

Bruno Carlos Da Cunha Costa

Angelo Ernani Maia Ciarlini

Fabio A M Porto

Fábio A Perosi

Amir H K Moghadam

Jonas F Bias

Luiz Guilherme Oliveira Dos Santos

Timothy A Voyt

Silvana Rossetto

Renato M M Medeiros

Vinícius Michel Göttin

Fábio A M Porto

Franklin Jordan Ventura Quico

Franklin Ventura

The quality of decompressed data that was compressed in a lossy manner to generate lossy compressed data, is improved. Lossy compressed data may be generated at a device and transmitted to a gateway. The gateway includes a reconstruction engine that is trained to reduce the compression loss by generating a learned residue. The residue is added to the lossy decompressed data to generated corrected decompressed data that has a smaller loss than the lossy decompressed data.

Quality of lossy compressed sensor data with machine learning

Reconstructing compressed data to reduce compression loss. Data at a device is compressed using lossy compression and metadata values are added to the compressed data. A gateway receives a package including the compressed data and the metadata values. The data is decompressed and input to a machine learning model along with the metadata values. The machine learning model is trained to reduce the compression loss. The output of the model is an improved decompressed data. Actions may be performed based on the improved decompressed data.

Reducing the compression error of lossy compressed sensor data

Deep reinforcement learning techniques and provenance-based simulation are employed for resource allocation in a shared computing environment. One method comprises: obtaining a specification of a workflow of concurrent workflows in a shared computing environment, wherein the specification comprises workflow states and one or more control variables for the workflow in the shared computing environment; obtaining a simulation model of the workflow representing different configurations of the control variables; evaluating the control variables for the concurrent workflows using a reinforcement learning (RL) agent by observing the states and obtaining an expected utility score for control variable combinations for the execution of the concurrent workflows given an allocation of resources of the shared computing environment, wherein the RL agent performs, using the simulation model, the evaluating, the obtaining the expected utility score, and/or a training of an RL model; and providing an allocation of the resources based on the expected utility score.

Deep reinforcement learning for workflow optimization using provenance-based simulation

Deep reinforcement learning techniques are provided for resource allocation in a shared computing environment. An exemplary method comprises: obtaining a specification of a workflow of a plurality of concurrent workflows in a shared computing environment, wherein the specification comprises a plurality of workflow states and one or more control variables for the workflow in the shared computing environment; evaluating values of the control variables for an execution of the concurrent workflows using a reinforcement learning agent by (i) observing the states, including a current state, and (ii) obtaining an expected utility score for combinations of the control variables for the execution of the concurrent workflows given an allocation of resources of the shared computing environment corresponding to the combination of control variables in the current state; and providing an allocation of the resources of the shared computing environment reflecting the combination having the expected utility score that satisfies a predefined score criteria.

Deep reinforcement learning for workflow optimization

Reinforcement learning is used to dynamically tune cache policy parameters. The current state of a workload on a cache is provided to a reinforcement learning process. The reinforcement learning process uses the cache workload characterization to select an action to be taken to adjust a value of one of multiple parameterized cache policies used to control operation of a cache. The adjusted value is applied to the cache for an upcoming time interval. At the end of the time interval, a reward associated with the action is determined, which may be computed by comparing the cache hit rate during the interval with a baseline hit rate. The process iterates until the end of an episode, at which point the parameters of the cache control policies are reset. The episode is used to train the reinforcement learning policy so that the reinforcement learning process converges to a trained state.

Using reinforcement learning to dynamically tune cache policy parameters

An information handling system for managing a storage system includes storage for storing profile-based cache policy performance prediction models. The information handling system also includes a storage manager that obtains an input-output profile for a workload hosted by the information handling system during a first period of time; obtains performance metrics for cache policies for the storage system using: the input-output profile, and the profile-based cache policy performance prediction models; obtains a ranking of the cache policies based on the performance metrics; selects a cache policy of the cache policies based on the rankings; and updates operation of a cache of the storage system based on the selected cache policy for a second period of time.

System and method for cache management

A method for optimizing performance of a storage system includes creating a structured state index from a universe of I/O traces of memory access operations in a storage system. The structured state index is validated against a target metric operational parameter of the storage system. If the structured state index has correlation against the target metric operational parameter of the storage system, the structured state index is used as input to a decision-making task. The decision-making task may be implemented as a deep neural network and the structured state index is used as input training data for the deep neural network. Once the decision-making task has been trained using the structured state index, the decision-making task is used in a predictive manner to generate a predicted target metric operational parameter of the storage system given a proposed storage policy.

Method and apparatus for optimizing performance of a storage system

Techniques are provided for workflow simulation using provenance data similarity and sequence alignment. An exemplary method comprises: obtaining a state of workflow executions of concurrent workflows with multiple resource allocation configurations, wherein the state comprises provenance data of the concurrent workflows; obtaining execution traces of the concurrent workflows representing different resource allocation configurations; identifying a set of states in a first execution trace and a set of states in a second execution trace as corresponding anchor states; mapping a first intermediate state to a second intermediate state between a pair of anchor states using the provenance data; generating a simulation model of the workflow executions representing the different configurations of the resource allocation; and generating new simulation traces of the workflow executions with resource allocation configurations that are not represented in the provenance data.

Workflow simulation using provenance data similarity and sequence alignment

A cache management system includes a sequentiality determination process configured to determine sequentiality profiles of a workload of IO traces as the workload dynamically changes over time. A learning process is trained to learn a correlation between workload sequentiality and cache pollution, and the trained learning process is used to predict cache pollution before the cache starts to experience symptoms of excessive pollution. The predicted pollution value is used by a cache policy adjustment process to change the prefetch policy applied to the cache, to proactively control the manner in which prefetching is used to write data to the cache. Selection of the cache policy is implemented on a per-LUN basis, so that cache performance for each LUN is individually managed by the cache management system.

Method and apparatus for adjusting cache prefetch policies based on predicted cache pollution from dynamically evolving workloads

Techniques are provided for characterizing and quantifying a sequentiality of workloads using sequentiality profiles and signatures. One exemplary method comprises obtaining telemetry data for an input/output workload; evaluating a distribution over time of sequence lengths for input/output requests in the telemetry data by the input/output workload; and generating a sequentiality profile for the input/output workload to characterize the input/output workload based at least in part on the distribution over time of the sequence lengths. Multiple sequentiality profiles for one or more input/output workloads may be clustered into a plurality of clusters. A sequentiality signature may be generated to represent one or more sequentiality profiles within a given cluster. A performance of data movement policies may be evaluated with respect to the sequentiality signature of the given cluster.

Sequentiality characterization of input/output workloads

Techniques are provided for detecting resource bottlenecks in workflow task executions using provenance data. An exemplary method comprises: obtaining a state of multiple workflow executions of multiple concurrent workflows performed with different resource allocation configurations in a shared infrastructure environment; obtaining first and second signature execution traces of a task representing first and second resource allocation configurations, respectively; identifying first and second corresponding sequences of time intervals in the first and second signature execution traces for the task, respectively, based on a similarity metric; and identifying a given time interval as a resource bottleneck of a resource that differs between the first and second resource allocation configurations based on a change in execution time for the given time interval between the first and second signature execution traces. The first signature execution trace may be obtained by disaggregating data related to batches of workflow executions.

Detection of resource bottlenecks in execution of workflow tasks using provenance data

Techniques are provided for simulation-based online workflow optimization. One method comprises obtaining a state of concurrent workflows; obtaining state similarity functions that assign a similarity score between pairs of states; generating a simulation model of the workflow executions representing different configurations of at least one control variable in the workflow executions by mapping states with a highest similarity; obtaining at least one utility function that assigns a utility score to the states in the simulation model; determining a configuration of the control variable that maximizes the utility score for states in the simulation model; and for new concurrent workflows: obtaining a current state of the new concurrent workflows; identifying a most similar state with one of the determined configurations of the control variable in the simulation model with a highest similarity to the current state; and adjusting the configuration of the control variable of the new concurrent workflows to match the corresponding configuration of the control variable of the most similar state.

Simulation-based online optimization of workflows

Techniques are provided for predicting a time-to-finish of at least one workflow in a shared computing environment using a deep neural network with a biangular activation function. An exemplary method comprises: obtaining a specification of an executing workflow of multiple concurrent workflows in a shared computing environment, wherein the specification comprises states of past executions of the executing workflow; obtaining a trained deep neural network, wherein the trained deep neural network is trained to predict one or more future states of the executing workflow using the states of past executions and wherein the trained deep neural network employs a biangular activation function comprising multiple parameters that define a position and a slope associated with two angles of the biangular activation function for a range of input values; and estimating, using the at least one trained deep neural network, a time-to-finish of the executing workflow of the multiple concurrent workflows.

Predicting time-to-finish of a workflow using deep neural network with biangular activation functions

Pattern queries are evaluated in parallel over large N-dimensional datasets to identify features of interest. Similarity-based pattern matching tasks are executed over N-dimensional input datasets comprised of numeric values by providing data representations for the N-dimensional input datasets, a pattern query and one or more candidate solutions for the pattern query, such that the pattern query specifies a pattern of an N-dimensional body that is compared to at least one candidate solution corresponding to an N-dimensional body extracted from the N-dimensional input datasets; defining a distance metric that compares the N-dimensional body formed by the candidate solution extracted from the N-dimensional input datasets and the N-dimensional body formed by the pattern query, taking into account one or more of the following criteria: differences between mapped values, differences in scale and differences in shape; and executing, in parallel, a plurality of independent instances of at least one algebraic operator to generate and score the candidate solutions based on the distance metric. The exemplary algebraic operators comprise a Trace Match operator, a Ranking operator, a Candidate Solution operator, and a Query Clustering operator.

Methods and apparatus for parallel evaluation of pattern queries over large N-dimensional datasets to identify features of interest

Lossless content-aware compression and decompression techniques are provided for floating point data, such as seismic data. A minimum-length compression technique exploits an association between an exponent and a length of the significand, which corresponds to the position of the least significant bit of the significand. A reduced number of bits from the significand can then be stored. A prediction method is also optionally previously applied, so that residual values with shorter lengths are compressed instead of the original values. An alignment compression technique exploits repetition patterns in the floating point numbers when they are aligned to the same exponent. Floating point numbers are then split into integral and fractional parts. The fractional part is separately encoded using a dictionary-based compression method, while the integral part is compressed using a delta-encoding method. A prediction method is also optionally previously applied to the integral part, thereby increasing the compression ratio.

Growing community of inventors

Rio de Janeiro, Brazil

Alex Laier Bordignon