Emc IP Holding Company LLC

Vmware, Inc.

Emc Corporation

Ruiling Dou

Xianlong Liu

Wenguang Wang

Alexander S Mathews

Tao Xie

Changyong Yu

Chang Yong Yu

Philippe Armangau

Walter Forrester

Feng Zhang

Zhao Xia Jin

Ahsan Rashid

Ying Hu

Zongliang Li

Banghui Luo

Gyanesh Kumar Choudhary

Yuetao Jiang

Jean-Pierre Bono

Jun Hui Liu

Asit Desai

Richard P Ruef

Gong Chen

Kurt William Everson

Satish Pudi

Hao Pan

Qi Mao

Henry Hao Fang

Piers Yu

Kevin Rayfeng Li

Chandrakanth Gadhiraju

Xin Li

Litao Xia

Abhay Kuamr Jain

A method for encryption in a distributed datastore is provided. The method generally includes receiving random data from a virtualization management platform as a wrapped data encryption key (DEK), retrieving a key encryption key (KEK) from a key management server (KMS), decrypting the wrapped DEK using the KEK to determine a blank DEK, where the blank DEK is common to a plurality of hosts of a host cluster accessing a distributed datastore; encrypting first data using the blank DEK, and storing the encrypted first data in one or more disks of the distributed datastore, the one or more disks belonging to the plurality of hosts.

Encryption in a distributed storage system utilizing cluster-wide encryption keys

The disclosure herein describes converting a disk cluster to a different format. A format conversion instruction associated with a disk cluster is received. A first subgroup of disks of the disk cluster that are the emptiest disks of the disk cluster are identified and all data is evacuated from the first subgroup of disks to other disks of the disk cluster. The first subgroup of disks is reformatted based on the received format conversion instruction. A group of data objects stored in the disk cluster is converted based on the format conversion instruction and the converted group of data objects are written to the reformatted first subgroup of disks. The process iterates through the disks of the disk cluster to reformat all disks and convert all data objects based on the received format conversion instruction. The process reduces the write operations required to convert the format of the disk cluster.

Converting the format of a distributed object storage with reduced write operations

The present disclosure relates to bootstrapping an encrypted single node VSAN cluster. One method includes receiving a request to create an encrypted VSAN cluster from a single host in a software-defined datacenter, deploying a virtual server on a VSAN datastore of the software-defined datacenter, registering a native key provider (NKP) in the virtual server, creating an empty VSAN cluster encrypted by the NKP, adding the single host to the encrypted empty cluster to create a one-host encrypted cluster, registering a KMIP KMS in the virtual server, switching encryption of the one-host encrypted cluster from the NKP to the KMIP KMS, and adding another host to the one-host encrypted cluster to create the encrypted cluster.

Bootstrapping an encrypted single node VSAN cluster

An example method for a first host, being an owner of an object stored in a virtual storage area network (vSAN) cluster, to perform encryption and decryption operations during a rekey in the vSAN cluster is disclosed. The method includes obtaining a first encryption key and a first key identifier (ID) of the first encryption key; transmitting the first key ID and an active key index to a second host; using the first encryption key to perform encryption and decryption operations; and in response to a determination of receiving a key change notification from a master node of the vSAN cluster, terminating a connection with the second host.

Method and system to perform encryption and decryption operations during rekey in virtualized computing environment

A technique for managing storage space includes establishing an MSR (minimum space reservation) of a thin file system built upon a storage pool. In response to a set of storage requests to the file system to store new data, the method further includes obtaining an additional space guarantee from the storage pool. After deleting data from the file system such that the size of the file system falls below the MSR, the technique further includes performing a space reclaim operation, the space reclaim operation (i) compacting the file system to a size less than the MSR, (ii) canceling the additional space guarantee such that the storage pool no longer guarantees the additional space for the file system, and (iii) continuing to reserve the full MSR of the file system, even though the size of the file system is smaller than the MSR.

Enforcing minimum space guarantees in thinly-provisioned file systems

A data storage system manages use of a pool of secondary storage by internal file systems hosting logical data storage objects accessed by clients. A choose-and-mark-slices operation scans the file system to identify a subset of the slices as evacuatable slices to be de-provisioned and returned to the pool, the subset having a size exceeding a per-iteration limit observed by a separate evacuate-slices operation. The subset is recorded in an in-memory structure. Each iteration of the evacuate-slices operation (1) examines the in-memory structure to select slices for evacuation, no more than the per-iteration limit being selected, (2) moves data from the selected slices to the remaining slices of the set, and (3) subsequently (i) updates metadata of the file system to reflect the moving of the data and (ii) de-provisions the selected slices from the file-system to return them to the pool for subsequent re-provisioning.

Data storage system using in-memory structure for reclaiming space from internal file system to pool storage

A method includes (a) while processing data storage requests from users directed to filesystems of a plurality of filesystems served from a data storage system, receiving a request to execute a filesystem reorganization task on a filesystem of the plurality of filesystems, (b) in response to receiving the request, checking whether the filesystem reorganization task has both a global reorganization token and a filesystem-specific reorganization token, the filesystem-specific reorganization token being specific to the filesystem on which the filesystem reorganization task has been requested to execute, and (c) in response to determining that the filesystem reorganization task does have both the global reorganization token and the filesystem-specific reorganization token, executing the filesystem reorganization task to move a data block from one location within the filesystem to another location within the filesystem. Other embodiments are directed to corresponding apparatuses, computer program products, and systems for performing similar methods.

Techniques for efficiently performing filesystem reorganization tasks for different filesystems

A technique for performing storage tiering accesses allocation metadata in a data storage system and applies that allocation metadata when relocating data from a selected extent to a target extent. The selected extent includes a range of contiguous blocks. The allocation metadata may designate each of these blocks as either allocated or free. When relocating data from the selected extent to the target extent, the technique copies data of the selected extent on a per-block basis, checking whether that block is allocated or free before copying it to the target extent.

Mitigating flash wear when performing storage tiering

Techniques for improving storage efficiency of a data storage system are provided. The techniques include (a) performing a testing operation to evaluate whether a test condition is satisfied for a filesystem having a plurality of subspaces, each subspace being provisioned with a respective amount of storage, each subspace of the filesystem being dedicated to storing different respective types of filesystem content, wherein the test condition indicates that a first subspace has available space in excess of a threshold percentage, (b) in response to determining that the test condition is satisfied for the filesystem, performing a reduction operation on the first subspace to remove a portion of storage provisioned to that subspace while retaining the removed portion for use by the filesystem, and (c) upon a second subspace reaching a fullness threshold, provisioning the removed portion to the second subspace and storing content of the respective type of the second subspace thereon.

Techniques for rebalancing storage between subspaces

A data storage system manages a pool of secondary storage by internal file systems hosting logical data storage objects accessed by clients, the pool organized into fixed-size, dynamically provisioned slices. A first choose-and-mark-slices operation scans a set of slices to identify a first subset as evacuatable, and records them as a recorded subset in an in-memory structure. Iterations of an evacuate-slices operation (1) select slices from among the recorded subset, observing a per-iteration limit, (2) move data from the selected slices to remaining slices of the set, and (3) subsequently (i) updates file system metadata to reflect the data movement and (ii) de-provisions the selected slices and returns them to the pool. Between iterations, a change in the composition of evacuatable slices is detected, and a second choose-and-mark-slices operation re-scans the slices and records a second subset as the recorded subset for a subsequent iteration of the evacuate-slices operation.

Data storage system dynamically re-marking slices for reclamation from internal file system to pool storage

A computer-implemented technique performs block deduplication. The technique involves generating a list of candidate blocks and adding classifications to the list of candidate blocks. The classifications classify the candidate blocks on the list into different categories based on a set of block sequence matching criteria. The technique further involves performing, based on the classifications added to the list of candidate blocks, further block deduplication processing on first candidate blocks having a first classification while not performing further block deduplication processing on second candidate blocks having a second classification. Examples of the categories include an 'exact long match' category, a 'disrupted long match' category, a &#x201c;loosely long match&#x201d; category, and a &#x201c;short match&#x201d; category.

Performing block deduplication using block sequence classifications

Techniques for shrinking a filesystem backed by a volume identify slices in the volume to be evacuated in order to reach a target size, identify a target endpoint in the volume, and evacuate identified slices to available locations prior to the target endpoint. The same data is typically not moved from slice to slice multiple times.

Efficiently shrinking a dynamically-sized volume

Improved techniques of implementing range lock involve granting a sleeping thread access to an address range when no conflicts exist with earlier pending requests as well as with already granted requests. Along these lines, a current child thread that has a conflict with a parent thread that currently holds a range lock on a range of bytes during a read/write operation will be awoken from waiting state when the parent thread has completed its access and releases its lock. In response, the processor compares the range to which the current child thread request access with the ranges to which other sleeping threads request access. However, the comparisons are only performed in connection with (i) requests that arrived prior to the current child thread and (ii) other granted requests.

Growing community of inventors

Shanghai, China

Ruiling Dou