Netapp, Inc.

Palak Sharma

Cheryl Marie Thompson

Qinghua Zheng

Venkateswarlu Tella

Sindhushree K N

Dibyasri Nandi

Debanjan Paul

Dinakaran Narayanan

Techniques are provided for mirroring objects between object stores hosted by cloud providers that could have different data layout requirements. An object may be stored within an object store that supports a unified object format where the object is capable of storing compressed data. The object may be mirrored to a destination object store that may also support the unified object format or to a destination object store that does not support the unified object format. If the destination object store does not support the unified object format, then slot header metadata within the object is used to decompress the data within the object into an uncompressed format. The data is packaged from being in the uncompressed format into a fixed offset format supported by the destination object store to create a mirrored object that is stored into the destination object store while retaining compression of the data.

Mirroring objects between different cloud providers

Techniques are provided for mirroring objects between object stores hosted by cloud providers that have different data layout requirements. An object may be stored within a first object store that supports a fix offset format where uncompressed data is stored according to fixed offsets and boundaries within fixed size objects. A mirroring operation may be used to mirror the object to a second object store that supports a unified object format where compressed data can be stored at non-fixed offsets and boundaries within variable sized objects. The mirroring operation selects a compression algorithm and compresses the object on the fly to create a mirrored object having the unified object format. The mirrored object, populated with the compressed data and slot header metadata comprising compression information for how to locate and decompress the data in the mirrored object, is stored into the second object store.

Mirroring objects between different cloud providers with different data layout requirements

Techniques are provided for implementing a unified object format. The unified object format is used to format data in a performance tier (e.g., infrequently accessed data, snapshot data, etc.) into objects that are stored into an object store for low cost, scalable, long term storage compared to storage of the performance tier. With the unified object format, compression of the data may be retained when the data is stored as the objects into the object store. Additional compression may also be provided for the data in the objects. The unified object format includes slot header metadata used to track the location of the data within the object notwithstanding the data being compressed and/or stored at non-fixed boundaries. The slot header metadata may be cached at the performance tier for improved read performance and may be repaired by a repair subsystem (a slot header repair subsystem).

Persistent metafile used to cache slot header metadata for improved read performance of objects within an object store

Repair subsystem for persistent metafile repair to improve resiliency

Unified object format for retaining compression and performing additional compression for reduced storage consumption in an object store

Growing community of inventors

Haryana, India

Palak Sharma