Vmware, Inc.

Oracle America, Inc.

Oracle International Corporation

Ilia Sokolinski

Wenguang Wang

Junlong Gao

Maxime Austruy

Petr Vandrovec

Satish Pudi

Satyam B Vaghani

Vamsi Gunturu

Ilya Languev

Komal Desai

Chiao-Chuan Shih

Sanjay Vasudev Acharya

Tejasvi Aswathanarayana

Rajesh Bhat

Howard Hall

Milan Merhar

Sujay Shrikant Godbole

Ilia Langouev

Patrick William Penzias Dirks

Jesse Pool

Derek Uluski

Vamsidhar Gunturu

In one set of embodiments, a computer system can create a snapshot of a data set, where the snapshot includes a plurality of data blocks of the data set. The computer system can upload the snapshot to a cloud object storage platform of a cloud infrastructure, where the snapshot is uploaded as a plurality of log segments conforming to an object format of the cloud object storage platform, and where each log segment includes one or more data blocks in the plurality of data blocks, and a set of metadata comprising, for each of the one or more data blocks, an identifier of the data set, an identifier of the snapshot, and a logical block address (LBA) of the data block. The computer system can then communicate the set of metadata to a server component running in a cloud compute and block storage platform of the cloud infrastructure.

Recovering the metadata of data backed up in cloud object storage

The present disclosure is related to methods, systems, and machine-readable media for supporting deduplication in file storage using file chunk hashes. A hash of a chunk of a log segment can be received from a software defined data center. A chunk identifier can be associated with the hash in a hash map that stores associations between sequentially-allocated chunk identifiers and hashes. The chunk identifier can be associated with a logical address corresponding to the chunk of the log segment in a logical map that stores associations between the sequentially-allocated chunk identifiers and logical addresses. A search of the hash map can be performed to determine if the chunk is a duplicate, and the chunk can be deduplicated responsive to a determination that the chunk is a duplicate.

Supporting deduplication in file storage using file chunk hashes

Solutions for managing archived storage include receiving, at a first node, a snapshot comprising object data (e.g., a virtual machine disk snapshot) from a second node (e.g., a software defined data center), and storing the snapshot in a tiered structure that includes a data tier and a metadata tier. Snapshots may be used for fail-over operations and/or backups, to support disaster recovery. The data tier comprises a log-structured file system (LFS), and the metadata tier comprises a content addressable storage (CAS) identifying addresses within the LFS. The metadata tier also comprises a logical layer indicating content in the CAS. Segment cleaning of the data tier is performed using a segment usage table (SUT). Some examples include performing a fail-over operation from the second node to a third node using at least the stored snapshot for workload recovery. In some examples, the CAS comprises a log-structured merge-tree (LSM-tree).

Log-structured formats for managing archived storage of objects

The present disclosure is related to methods, systems, and machine-readable media for supporting deduplication in object storage using subset hashes. A plurality of hashes of a plurality of blocks of a plurality of log segments can be received from a software defined data center, wherein each block corresponds to a respective logical address. Each of the plurality of logical addresses can be associated with a respective sequentially-allocated chunk identifier in a logical map. A subset hash comprising a hash of a subset of the plurality of blocks can be determined that corresponds to a contiguous range of the plurality of logical addresses. A search of a hash map for the subset hash can be performed to determine if the subset hash is a duplicate. The subset of the plurality of blocks can be deduplicated responsive to a determination that the subset hash is a duplicate.

Supporting deduplication in object storage using subset hashes

Techniques for the increased efficiency of storing data objects storage in the object storage of a software designed data center (SDDC) are provided. The techniques include the efficient storage of data, while enabling snapshots of each updating of the data. The snapshots of the data may be efficiently recovered via the techniques. Difference-level mappings for each snapshot are encoded in compact self-balancing data trees included in the object's metadata. The metadata mappings include mappings between various address spaces employed by the SDDC, as well as the address spaces employed by data stores that store the data on physical medium. Because the metadata is efficiently structured, the metadata for an object may be cached for quick lookups during data access and/or snapshot recovery. The techniques also provide low-latency recovery and/or system rollback in the event of any failure in the SDDC.

Distributed object storage supporting difference-level snapshots

Techniques for tiering data to a cold storage tier of a cloud object storage platform are provided. In one set of embodiments, a computer system can identify one or more old snapshots of a data set that reside in a first storage tier of the cloud object storage platform, where the one or more old snapshots are snapshots that are unlikely to be deleted from the cloud object storage platform within a period of N days. The computer system can further, for each snapshot in the one or more old snapshots: identify one or more data blocks in the snapshot that are superseded by a more recent snapshot in the one or more old snapshots; write the one or more data blocks to a second (i.e., cold) storage tier of the cloud object storage platform that has a lower storage cost than the first storage tier; and cause the one or more data blocks to be deleted from the first storage tier.

Tiering data to a cold storage tier of cloud object storage

A method for managing metadata for data stored in a cloud storage is provided. The method receives, at a first of a plurality of metadata servers, information associated with an object stored in the cloud storage, the information comprising a plurality of LBAs for where the object is stored. Each metadata server allocates contiguous chunk IDs for a group of objects. The method generates a new chunk ID for the object, which is a combination of a unique fixed value and a monotonically incrementing local value associated with each LBA, such that a first LBA is mapped to a first chunk ID having a first local value and a next LBA is mapped to a second chunk ID having the first local value incremented as a second local value. The method stores the new chunk ID and other metadata in one or more tables stored in a metadata storage.

Managing metadata for a backup data storage

A method for accelerating TCP/UDP packet switching. The method involves determining whether exception processing is necessary; if not, the packet is forwarded to a special stack for expedited processing. Packets requiring exception processing are forwarded to the conventional stack.

TCP/UDP acceleration

A framework for converting between copy-on-write (COW) and redo-based technologies is disclosed. To take a virtual disk snapshot, disk descriptor files, which include metadata information about data stored in virtual volumes (vvols), are 'swizzled' such that the descriptor file for a latest redo log, to which IOs are currently performed, points to the base vvol of a COW-based vvol hierarchy. A disk descriptor file previously associated with the base vvol may also be updated to point to the vvol newly created by the snapshot operation. To revert to an earlier disk state, a snapshot may be taken before copying contents of a snapshot vvol of the COW-based vvol hierarchy to a base vvol of the hierarchy, thereby ensuring that the reversion can be rolled back if it is unsuccessful. Reference counting is performed to ensure that vvols in the vvol hierarchy are not orphaned in delete and revert use cases. Differences between vvols in the COW-based vvol hierarchy are used to clone the hierarchy and to migrate the hierarchy to a redo-based disk hierarchy.

Conversion of virtual disk snapshots between redo and copy-on-write technologies

The storage system exports logical storage volumes that are provisioned as storage objects. These storage objects are accessed on demand by connected computer systems using standard protocols, such as SCSI and NFS, through logical endpoints for the protocol traffic that are configured in the storage system. Logical storage volumes are created from a logical storage container having an address space that maps to storage locations of the physical data storage units. Each of the logical storage volumes so created has an address space that maps to the address space of the logical storage container. A logical storage container may span more than one storage system and logical storage volumes of different customers can be provisioned from the same logical storage container with appropriate security settings.

Object storage system

The storage system exports logical storage volumes that are provisioned as storage objects. These storage objects are accessed on demand by connected computer systems using standard protocols, such as SCSI and NFS, through logical endpoints for the protocol traffic that are configured in the storage system. When traffic through a particular protocol endpoint is busy, the logical storage volume undergoes a rebind process pursuant to which the logical storage volume is bound to a new protocol endpoint, and the binding to the old protocol endpoint is removed upon completion of all pending input-output commands issued through the old protocol endpoint.

Method of balancing workloads in object storage system

The storage system exports logical storage volumes that are provisioned as storage objects within logical storage containers. These storage objects are accessed on demand by connected computer systems using standard protocols, such as SCSI and NFS, through logical endpoints for the protocol traffic that are configured in the storage system. A snapshot profile can be separately defined for each of these logical storage volumes, even for those that are within the same storage container. The snapshot profile for a logical storage volume defines whether or not snapshot is enabled for that logical storage volume, the frequency of the snapshot, and the number of snapshots to be retained.

Growing community of inventors

Brighton, MA, United States of America

Ilia Sokolinski