Emc IP Holding Company LLC

Eduardo Vera Sousa

Tiago Salviano Calmon

Vinícius Michel Gottin

Hugo De Oliveira Barbalho

Vinicius Michel Gottin

Rômulo Teixeira De Abreu Pinho

Jonas Furtado Dias

Alex Laier Bordignon

Daniel Sadoc Menasché

Manuel Ramón Vargas Avila

Owen Martin

Adriana Bechara Prado

Roberto Nery Stelling Neto

Edward José Pacheco Condori

Romulo D Pinho

Amit Bhaya

Oumar Diene

Jonathan Ferreira Passoni

Percy Rivera Salas

Techniques described herein relate to systems and methods for workload placement based on subgraph similarity. Such techniques may include obtaining an encoded workload graph based on receiving a workload execution request; using the encoded workload subgraph to obtain encoded graphs representing previous workload executions, encoded subgraphs representing infrastructures on which the workload were executed, resource usage information, and execution metrics; using the encoded infrastructure subgraphs using subgraph similarity to identify candidate infrastructure subgraphs, using an ML model to predict an execution metric for an execution of the workload using the candidate; and selecting a best candidate infrastructure on which to execute the workload based on the predicted execution results.

Systems and methods for workload placement based on subgraph similarity

Techniques described herein relate to a method for resource allocation using fingerprint representations of telemetry data. The method may include receiving, at a resource allocation device, a request to execute a workload; obtaining, by the resource allocation device, telemetry data associated with the workload; identifying, by the resource allocation device, a breakpoint based on the telemetry data; identifying, by the resource allocation device, a workload segment using the breakpoint; generating, by the resource allocation device, a fingerprint representation using the workload segment; performing, by the resource allocation device, a search in a fingerprint catalog using the fingerprint representation to identify a similar fingerprint; obtaining, by the resource allocation device, a resource allocation policy associated with the similar fingerprint; and performing, by the resource allocation device, a resource policy application action based on the resource allocation policy.

Mapping telemetry data to states for efficient resource allocation

Techniques are provided for allocating resources for one or more workloads. One method comprises obtaining a current performance of a workload; determining an adjustment to a current allocation of a resource allocated to the workload by evaluating a representation of a relationship between: (i) the current allocation of the resource allocated to the workload, (ii) a performance metric, and (iii) the current performance of the workload; and initiating an application of the determined adjustment to the current allocation of the resource for the workload. The performance metric may comprise a nominal value of a predefined service metric and the current performance of the workload may comprise a current value of a variable that tracks a given predefined service metric of the workload. An amount (or percentage) of the adjustment permitted for each iteration may be controlled. A sum of allocated resources can be constrained to an amount of available resources.

Resource adaptation using nonlinear relationship between system performance metric and resource usage

Techniques are provided for assigning workloads in a multi-node processing environment using resource allocation feedback from each node. One method comprises obtaining feedback from distributed nodes that process workloads, wherein the feedback for a given node indicates (i) an allocation of resources, and (ii) a number of executing workloads. In response to receiving a given workload to be processed, candidate nodes are identified to execute the given workload; and the given workload is assigned to a given candidate node based on an amount of available resources on each candidate node and/or a stability of resource adjustments made for each candidate node. The stability of the resource adjustments made for each candidate node can be evaluated based on a maximum resource adjustment made for a given candidate node relative to a maximum resource adjustment made for each of the candidate nodes.

Assigning workloads in a multi-node processing environment using feedback from each node

Techniques are provided for adaptive resource allocation for multiple workloads. One method comprises obtaining a dynamic system model based on a relation between an amount of a resource for multiple iterative workloads and a predefined service metric; obtaining an instantaneous value of the predefined service metric; applying to a given controller associated with a given iterative workload of the multiple iterative workloads: (i) the dynamic system model, (ii) an interference effect of one or more additional iterative workloads on the given iterative workload, and (iii) a difference between the instantaneous value and a target value for the predefined service metric. The given controller applies an adjustment to the amount of the resource for the given iterative workload based at least in part on the difference. The resource allocation for the multiple iterative workloads can be performed in a sequence substantially in parallel with an execution of the iterative workloads.

Adaptation of resource allocation for multiple workloads using interference effect of resource allocation of additional workloads on performance

One example method includes accessing I/O traces, generating parameters based on the I/O traces, and defining an autoencoder deep neural network, training the autoencoder deep neural network using the parameters, collecting and storing new I/O traces, computing an encoded features difference series using the new I/O traces, detecting breakpoints in the encoded features difference series, evaluating a utility of the breakpoints, and performing an action based on the breakpoint utility evaluation.

Adaptable online breakpoint detection over I/O trace time series via deep neural network autoencoders re-parameterization

Techniques are provided for predicting a time to complete a data protection operation. One method comprises obtaining metadata for (i) a given data protection appliance, and/or (ii) a cluster of similar data protection appliances comprising the given data protection appliance; evaluating first level features using the obtained metadata; evaluating a second level feature using some of the evaluated first level features; and processing one or more of the first level features, and the second level feature, using a model that provides a predicted time to complete a data protection operation with respect to data of a protected device associated with the given data protection appliance. The predicted time may comprise a tolerance based on a robustness factor. The predicted time may be based on a number of protected devices that are concurrently undergoing a data protection operation with the protected device for one or more time intervals.

Artificial intelligence techniques for prediction of data protection operation duration

One example method includes identifying a group of asset backups to be performed, and each asset backup is associated with a respective asset and has an associated backup time and RPO, selecting an asset backup to run first, and the asset backup that will run first is chosen based on a start deadline of that asset backup relative to respective start deadlines of one or more other asset backups, and the start deadline falls within a time slot, selecting a stream from a group of streams for the selected asset backup, and the selected stream is a stream with a lowest value of first available time slot, and backing up the asset at a backup server by running the selected asset backup, and backup begins at a start time that is a time when the selected stream becomes available, and the asset backup runs on the selected stream.

Efficient method to build a backup scheduling plan for a server

A backup orchestrator for providing backup services to entities includes storage for storing recovery point objectives for the entities and a backup manager. The backup manager selects an optimization periodicity based a number of backups to be generated to meet a portion of the recovery point objectives; makes a determination that at least one of the portion of the recovery point objectives has a maximum allowable unbacked up period of time that is greater than the optimization periodicity; in response to the determination: load balances the number of backups across multiple optimization periods, based on the optimization periodicity, of a balanced backup schedule; selects a backup generation time for each of the to be generated backups in each of the optimization periods of the balanced backup schedule; and generates the number of backups using the balanced backup schedule.

Iterative integer programming with load balance for cyclic workloads

An efficient method for generating asset backup scheduling plans. Within a data protection environment, at least two service level metrics may be observed&#x2014;a recovery point object (RPO) and a recovery time objective (RTO). In order to meet acceptable values for these metrics, on par with established service level agreements, infrastructure employed throughout the data protection environment, as well as the scheduling of asset backup operations, tend to grow in complexity. To address service distributions potentially emerging from the aforementioned complexities, the disclosed method proposes a heuristic approach to generating asset backup scheduling plans, which consider factors such as backup device limitations, RPO violation minimization, asset usage, and asset prioritization.

Efficient method for generating asset backup scheduling plans

A method for managing storage devices includes obtaining a storage device cluster request, and in response to the storage device cluster request: obtaining a set of storage device telemetry entries associated with a plurality of storage devices, performing a telemetry normalization on the storage device telemetry entries to obtain a set of normalized entries, performing a pairwise evaluation on the set of normalized entries to obtain a set of initial storage device clusters, wherein a storage device cluster in the set of initial storage device clusters comprises a portion of the plurality of storage devices, performing a cluster re-evaluation on the set of initial storage device cluster groups to obtain a set of updated storage device clusters, updating a backup policy based on the set of updated storage device cluster groups, and performing a backup operation on a storage device based on the backup policy.

System and method for managing backup operations of storage devices in a backup storage system using cluster evaluations based on usage telemetry

Resource allocation to workloads is disclosed. Telemetry data associated with existing or previously executed workloads is stored and used to develop models. Telemetry data from new workloads are collected and, using the models, a fingerprint is extracted and compared to the fingerprints of previous workloads. This allows the initial allocation of resources to the new workload to be improved and aids in resource allocation convergence.

Dynamic resource allocation based on fingerprint extraction of workload telemetry data

Techniques are provided for adaptive resource allocation for workloads with initial condition setting. One method comprises obtaining a dataset comprising data from previous executions of a workload, wherein the data comprises a plurality of different resource allocations and parameterizations of the workload; determining an initial allocation of an amount of a resource for the workload based on a regression model characterizing a behavior of the workload, the data, a predefined service metric and a characterization of a target infrastructure; and initiating an application of the determined initial allocation of the amount of the resource for the workload. A performance of one or more of the plurality of workloads can be evaluated based on a percentage of time within a predefined error range. The regression model can be updated and/or replaced over time with new data for additional executions of the at least one workload.

Initialization of resource allocation for a workload characterized using a regression model

Techniques are provided for workflow simulation using provenance data similarity and sequence alignment. An exemplary method comprises: obtaining a state of workflow executions of concurrent workflows with multiple resource allocation configurations, wherein the state comprises provenance data of the concurrent workflows; obtaining execution traces of the concurrent workflows representing different resource allocation configurations; identifying a set of states in a first execution trace and a set of states in a second execution trace as corresponding anchor states; mapping a first intermediate state to a second intermediate state between a pair of anchor states using the provenance data; generating a simulation model of the workflow executions representing the different configurations of the resource allocation; and generating new simulation traces of the workflow executions with resource allocation configurations that are not represented in the provenance data.

Workflow simulation using provenance data similarity and sequence alignment

Techniques are provided for detecting resource bottlenecks in workflow task executions using provenance data. An exemplary method comprises: obtaining a state of multiple workflow executions of multiple concurrent workflows performed with different resource allocation configurations in a shared infrastructure environment; obtaining first and second signature execution traces of a task representing first and second resource allocation configurations, respectively; identifying first and second corresponding sequences of time intervals in the first and second signature execution traces for the task, respectively, based on a similarity metric; and identifying a given time interval as a resource bottleneck of a resource that differs between the first and second resource allocation configurations based on a change in execution time for the given time interval between the first and second signature execution traces. The first signature execution trace may be obtained by disaggregating data related to batches of workflow executions.

Growing community of inventors

Niterói, Brazil

Eduardo Vera Sousa