Microsoft Technology Licensing, LLC

Ioan Oltean

Aaron William Ogus

Bradley Gene Calder

Shane Kumar Mainali

Ju Wang

Andrew James Edwards

Cheng Huang

Yikang Xu

Huseyin Simitci

Scott Chao-Chueh Lee

Vadim Makhervaks

Laura Marie Caulfield

Lee Edward Prewitt

Madhav Himanshubhai Pandya

Maneesh Sah

Rajsekhar Das

David Allen Goebel

Garret Buban

Jegan Devaraju

Rushi Srinivas Surla

Omar Carey

Bryan Stephen Matthew

Julia Johnstone

Zone hints for use with a zoned namespace (ZNS) storage device. Zone hints include one or more of a first hint indicating that a zone is part of a group of a plurality of zones, a second hint indicating that the zone is to be fast-filled, or a third hint indicating that the zone is associated with a background operation. The first hint is structured to instruct the ZNS storage device to allocate to the zone a first storage resources that are physically adjacent to second storage resources reserved for others of the plurality of zones. The second hint is structured to instruct the ZNS storage device to bypass a staging area when writing to the zone. The third hint is structured to instruct the ZNS storage device to deprioritizing at least one operation writing to the zone, or to bypass the staging area when writing to the zone.

Zone hints for zoned namespace storage devices

In various embodiments, methods and systems for implementing a distributed metadata management system in distributed storage systems are provided. A distributed storage system operates based on data storage resources (e.g., extents and streams). The distributed metadata management system is implemented for extent and stream metadata to facilitate the scalability of metadata processing. The distributed storage system implements extent managers and stream managers that independently manage extent and stream metadata, respectively. The extent managers are associated with an extent table that stores extent metadata. The stream managers are associated with streams that store associations with extents. The distributed metadata management system can also utilize a bootstrap layer that leverages components of a legacy distributed storage system to facilitate distributed management of extent and stream metadata. The bootstrap layer is used to store the extent table as a system table and to persist the state of the stream manager as system streams.

Distributed metadata management in a distributed storage system

Various methods and systems for in-place unformatting of disks are provided. The system includes a preparation component configured for creating a backup file on the disk and a preformatting component for using the backup file to occupy a predetermined location that defines a backup zone on the disk. The backup file is a space-holder such that the preformatting component copies primal data and file table data to the backup zone. The system further includes a formatting component for formatting the disk storing the primal data and the file table and an unformatting component that copies the primal data and file table data from the backup zone back to their original locations to unformat the disk to a preformat configuration. A safeguard component may also be implemented to ensure that the preformat component is invoked before the format component if the preparation component was executed on a volume.

In-place disk unformatting

In various embodiments, methods and systems for implementing garbage collection in distributed storage systems are provided. The distributed storage system operates based on independent management of metadata of extent and stream data storage resources. A hybrid garbage collection system based on reference counting garbage collection operations and mark-and-sweep garbage collection operations is implemented. An extent lifetime table that tracks reference weights and mark sequences for extents is initialized and updated based on indications from extent managers and stream managers, respectively. Upon determining that an extent is to be handed-off from weighted reference counting garbage collection operations to mark-and-sweep garbage collection operations, a reference weight field for the extent is voided and a mark sequence field of the extent is updated. The mark sequence field is updated with a latest global sequence number. The mark-and-sweep garbage collection operations are utilized to reclaim the extent when the extent is no longer referenced.

Hybrid garbage collection in a distributed storage system

An append-only streams capability may be implemented that allows the host (e.g., the file system) to determine an optimal stream size based on the data to be stored in that stream. The storage device may expose to the host one or more characteristics of the available streams on the device, including but not limited to the maximum number of inactive and active streams on the device, the erase block size, the maximum number of erase blocks that can be written in parallel, and an optimal write size of the data. Using this information, the host may determine which particular stream offered by the device is best suited for the data to be stored.

Append only streams for storing data on a solid state device

Provided is a system and method for converting active data identified by a garbage collection operation into erasure coded fragments. In one example, the method may include identifying data blocks in use and interspersed among garbage data blocks not in use in cloud storage based on a garbage collection operation, extracting object data from the identified data blocks in use into a data container while leaving object data of the garbage data blocks not in use, and fragmenting a predetermined amount of extracted object data stored within the data container, the fragmenting comprising converting the predetermined amount of object data into a plurality of fragments including data fragments storing portions of the data and parity fragments for reconstructing the data, and writing the plurality of fragments in a distributed manner among a plurality of storage nodes.

Garbage collection implementing erasure coding

Growing community of inventors

Redmond, WA, United States of America

Ioan Oltean