Emc IP Holding Company LLC

Dell Products Inc.

Emc Corporation

Druva Inc.

Emc Coropration

Ajay Potnis

Adnan Sahin

Amit Dharmadhikari

Kiran B Halwai

Maneesh Singhal

Shampavman Chenjeri Gururajarao

Bharadwaj R Sathavalli

Sairam Veeraswamy

Sairam Iyer

Rahul Bhagwat

Assaf Natanzon

Jean-Pierre Bono

Xiaoye Jiang

Milind Vithal Borate

Milena Bergant

Sitaram D Pawar

Michael E Specht

John Michael Hayden

Anat Parush Tzur

Somesh Jain

Miles Aram DeForest

Dixit Patel

Rashmi Shashidhar

Nishant Shivajirao Thorat

Nir Goldschmidt

Sudeep Jathar

Preethi Srinivasan

Vijay Srinivasan

Prahlad Nishal

Jean Pierre Bono

Sindhura Chamala

Ritesh Singh

Ashwin B Agrawal

Mrinalini Chavan

Ramrao Patil

Pavan Ym

A resource allocation map generator generates a resource allocation map for a current data backup and/or restore based on a mathematical model, real-time operating data corresponding to operating states of one or more resources, and historical data corresponding to data back-up and restore of one or more historical datasets. A resource allocation recommender generates a recommendation for resource allocation for the current data backup and/or restore based on the resource allocation map and a threshold value corresponding to a particular resource. A resource allocator dynamically initiates a change in resource allocation based on the generated recommendation.

System and method for dynamic resource allocation during data backup/restore

A data back-up system configured to back-up one or more data sets from one or more devices to a data back-up server by using a proxy pool is presented. The data back-up system includes a load-balancer configured to distribute the one or more data sets across the proxy pool. The load balancer includes a data receiver configured to receive types of files, number of files, and total size of each file in the one or more data sets; a load estimator configured to estimate a weighted average load of each data set based on the number of files, the total size of each file, a compressibility factor for each file type, and an encryption factor for each file type; and a load distributor configured to distribute the one or more data sets as a plurality of workloads across the proxy pool.

System and method for data back-up using a proxy pool

A storage cluster includes a group of data nodes having concurrent access to a shared filesystem. The shared filesystem is assigned to a first TLU of a first storage group. Other filesystems of the data nodes associated with the shared filesystem, such as snapshot copies of the shared filesystems and the root/configuration filesystems of the data nodes are assigned to TLUs of a second set of storage groups. The first storage group and the second set of storage groups are all associated with a Remote Data Replication (RDR) group for the storage cluster. An RDR facility is created between a storage array storing the shared filesystem and a backup storage array. The concurrently accessed shared filesystem and the filesystems of all data nodes accessing shared filesystem are replicated on the RDR facility to the backup storage array to enable failover of the storage cluster between sites.

Method and apparatus for replicating a concurrently accessed shared filesystem between storage clusters

A storage node that maintains a replica of a logical volume for use in response to a failover trigger includes a data node with volatile memory in which a filesystem and its metadata and a VDM and its metadata associated with the replica are maintained prior to the failover trigger. The storage node also includes a SAN node in which data associated with the replica is maintained. The data is maintained in a RW (read-write) state by the SAN node prior to the failover trigger. However, the replica is presented in a RO (read-only) state by the storage node prior to the failover trigger. The storage node changes the in-memory state of the filesystem and VDM to RW responsive to the failover trigger. Because the filesystem and its metadata and VDM and its metadata are already in memory and the data is in a RW state in block storage the failover is completed relatively quickly.

Reducing failover time between data nodes

Snaps are distributed among data nodes for load balancing and overload avoidance. A snap is initially associated with a first VNAS server on a first data node that hosts the primary volume being snapped. A second data node is selected based on loading. The second data node may be in the same or a different cluster. A second VNAS server is instantiated on the second data node. The snap is then mounted on the second VNAS server. The second VNAS server may be configured as the target for all reads to the snap. If the second data node is in a different cluster than the snap may be copied or moved from a first SAN node to a second SAN node. Multiple copies of snaps may be distributed, and individual snaps and copies may be associated with different SLOs.

Distribution of snaps for load balancing data node clusters

Techniques for performing a restoration may include: creating a first snapshot of a primary file system at a first point in time; and at a second point in time, performing first processing to restore at least a portion of the primary file system from the first point in time. The first processing may include: creating a second snapshot of the first snapshot of the primary file system; exporting the second snapshot as a first target device; mounting the first target device as a read-write checkpoint file system; performing file system consistency processing on the read-write checkpoint file system; and subsequent to performing the file system consistency processing on the read-write checkpoint file system, restoring at least a portion of the primary file system from the second snapshot using the read-write checkpoint file system. The file system consistency processing may include correcting metadata inconsistencies of the read-write checkpoint file system.

File system restoration using block-based snapshots and file system consistency check

Software Defined Network Attached Storage data backup is implemented using storage array synchronous data replication. A TLU is created for each user filesystem managed by a Virtual Data Mover (VDM). A VDM configuration filesystem is also created containing metadata describing a mapping of user file system TLU to physical storage resources on the storage array. Both the user filesystem TLUs and the VDM configuration filesystem are mapped to an asynchronous Remote Data Forwarding (RDF) group of the storage array, which is used by the storage array to replicate both the user filesystem TLUs and VDM configuration filesystem. Filesystem ID and export path information are reserved on the primary storage array and backup storage array when the RDF group is set up, to enable transparent failover from the primary to the backup node of selected VDMs.

Software defined network attached storage backup using storage array synchronous or asynchronous data replication

A first Software Defined Network Attached Storage (SDNAS) process on a first storage system forwards snapshots of a file system provided by the first storage system to a second SDNAS process on a second storage system. The snapshots are forwarded by the first SDNAS process on an Internet Protocol (IP) network, to enable the first and second SDNAS processes to maintain redundant access to data of the file system without requiring the first storage system and second storage system to be interoperable. The initial snapshot includes a full version of the file system and subsequent snapshots include changed portions of the file system. Where the snapshot identifies changed portions at the extent level of granularity, the first SDNAS process subdivides the extents into sub-blocks, and only transmits changed sub-blocks of the extents on the IP network.

File system replication between software defined network attached storage processes using file system snapshots

A method of dynamic binding and unbinding thin logical storage volumes to data volumes stored in a storage array includes creating a snapshot copy of a user file system, storing the snapshot copy of the user file system in a data volume in the storage array, and binding a logical storage volume to the data volume. The logical storage volume is then used, by a snapshot file system, to access the data volume. If the data volume is not accessed for a period of time, the logical storage volume is unbound from the data volume while maintaining the snapshot copy of the user file system in the data volume. If an access event later occurs on the data volume, a second logical storage volume is bound to the data volume and used by the snapshot file system to access the data volume in connection with the access event.

Method and apparatus for dynamic binding and unbinding thin logical storage volumes to snapshots of a file system

A method for defining local failover within a storage system using storage group containers includes creating a plurality set of storage groups from physical storage resources of a storage array of the storage system, and creating a masking view identifying a first subset of the storage groups and specifying that Thin Logical Units (TLUs) in the first subset of storage groups are visible only to a primary Software Defined Network Attached Storage (SDNAS) process executing on the storage system and to a backup SDNAS process executing on the storage system. The first set of storage groups includes a Virtual Data Mover (VDM) configuration/root TLU and all user data TLUs used by the primary SDNAS process to thereby enable the set of storage groups used by the SDNAS to be used as a container for local failover.

Method and apparatus for defining local failure using storage group containers

A storage system includes a storage array having physical storage resources, a Software Defined Network Attached Storage (SDNAS) process running in an emulation on the storage system, a user file system having user file system data stored in the storage array that is accessed by the SDNAS process via a first Thin Logical Unit (TLU) storage volume mounted on an emulation operating system, a snapshot copy of the user file system stored in the storage array, and control logic. In some embodiments, the control logic is configured to associate a logical view of the snapshot copy of the user file system to a second TLU, mount the second TLU on the emulation operating system as a snapshot copy file system, and replay pending transactions and sub-transactions of the user file system on the snapshot copy file system to bring the snapshot copy file system to a stable state.

Method and apparatus for mounting and unmounting a stable snapshot copy of a user file system

Disaster recovery is provided for Software Defined Network Attached Storage where the data backup is implemented using asynchronous remote data replication. An asynchronous replication facility is created between a primary storage array and a backup storage array. User filesystems being handled by a virtual data mover (VDM) and a configuration filesystem describing the location of the user filesystems on the primary storage array are replicated on the remote data facility to the backup storage array. During failover, all filesystems associated with the asynchronous remote data facility are failed over from a primary storage system associated with the primary storage array to a backup storage system associated with the backup storage array. Where an individual filesystem or individual VDM is to be failed over to the backup storage system, a separate asynchronous replication remote data facility is created to separately replicate data for the aspect to be individually failed over.

Disaster recovery for software defined network attached storage using storage array asynchronous data replication

Disaster recovery is provided for Software Defined Network Attached Storage where the data backup is implemented synchronous data replication. A synchronous replication remote data facility is created between a primary storage array and a backup storage array. User filesystems being handled by a virtual data mover (VDM) and a configuration filesystem describing the location of the user filesystems on the primary storage array are replicated on the remote data facility to the backup storage array. Individual filesystems, individual VDMs, or groups of VDMs may be selectively failed over from the primary storage array to the backup storage array.

Disaster recovery for software defined network attached storage using storage array synchronous data replication

Data Virtual Data Movers (VDM) are assigned to nodes of the storage cluster and a backup node is assigned for each data VDM. A system VDM on each node collects node statistics including operational parameters of the node and activity levels of the data VDMs on the node. A cluster manager collects the node statistics from each of the system VDMs and uses weighted collected node statistics to assign a node score to each node in the storage cluster. The cluster manager uses the node scores to identify possible data VDM movement combinations within the storage cluster by applying a set of hard rules and a set of soft rules to evaluate the possible data VDM movement combinations. If a VDM movement combination is selected, it is implemented by moving at least some of the data VDMs within the cluster to attempt to equalize node scores within the cluster.

Method and apparatus for load balancing virtual data movers between nodes of a storage cluster

Embodiments are described for detecting a data organization issue in a storage system by analyzing static and dynamic layout characteristics for data stored in the storage system, workload characteristics for applications utilizing the data, virtualization effects of the storage system on the organization of the data, input/output characteristics of processes storing the data in the storage system, and cache usage characteristics of the data. For fragmentation issues, the system uses the analysis to provide defragmentation advisories to reduce fragmentation by at least one of: re-layout or re-tiering of the data in the data storage system, or cache memory reconfiguration.

Growing community of inventors

Pune, India

Ajay Potnis