Microsoft Technology Licensing, LLC

Other

Rajsekhar Das

Omar Carey

David Allen Dion

William R Tipton

Vladimir Petter

Gor Nishanov

Mathew George

David Bruce Lomet

Scott Chao-Chueh Lee

Michael T Massa

Rod N Gamache

Sunita Shrivastava

Chesong Lee

Rohit Jain

Rushabh A Doshi

Cornel Emilian Rat

Karan Mehra

Surendra Verma

Philip Alan Bernstein

Andrea D'Amato

Neeraj Kumar Singh

Malcolm James Smith

Bryan Stephen Matthew

Tristan Charles Griffith

Norbert Paul Kusters

Neal Robert Christiansen

Aaron William Ogus

Octavian Tony Ureche

Philip A Bernstein

Paul Adrian Oltean

Garret Buban

Andrew M Herron

Vinod R Shankar

Vadim Makhervaks

Gregory Irving Thiel

Scott A Brender

Rajeev Y Nagar

Laura Marie Caulfield

Juan-Lee Pang

Lee Edward Prewitt

Ioan Oltean

Daniel Chan

Madhav Himanshubhai Pandya

Elden Christensen

Erik G Hortsch

Gregorio Maeso

Shao-Chuan Wang

Thothathri Vanamamalai

Erik Schmidt

Kurt Lawrence Friedrich

Arushi Aggarwal

Karel Danihelka

Some storage systems are configured with VDL (valid data length) type controls that are implemented on a per cluster basis and, in some instances, on a sub-cluster basis, rather than simply a per file basis. In some instances, per-cluster VDL metadata for the storage clusters is stored and referenced at the edge data volume nodes of a distributed network for the storage system rather than, and/or without, storing or synchronizing the per-cluster VDL metadata at a master node that manages the corresponding storage clusters for the different data volume nodes. Sequence controls are also provided and managed by the master node and synchronized with the edge data volume nodes to further control access to data contained in the storage clusters.

Systems and methods for managing data storage cluster synchronization

A processing system dynamically adjusts the size of root nodes in hierarchical data structures, such as B+ trees. Upon creation, a root node has a predefined initial size. When a request is received to insert a new record, a determination is made as to whether the size of the root node has reached a predefined maximum size. If the root node has not reached the predefined maximum size, the size of the root node is increased to accommodate the new record and the new record is added to the root node. If the root node has reached the predefined maximum size, one or more new child nodes are created, and records are moved from the root node to the new child nodes. The size of the root node is then reduced to a predefined minimum size.

Dynamically resizing root nodes of hierarchical data structures

The techniques disclosed herein enable systems to store file system metadata on append-only storage devices by the introduction of a metadata layout and relative references. To achieve this, a system analyzes a file metadata structure to select an approach to storing the metadata structure on the storage device. If the metadata structure is above a threshold size, it is separated into layers that are then appended to the storage device from lowest to highest. An offset is retrieved for appended layers and provided to upper-level layers that refer to them until the full structure is stored. For storing smaller pieces of metadata that refer to each other, the system configures upper-level nodes with relative references that express the location of metadata relative to the node. This enables atomic write operations for smaller metadata structures and/or portions of larger structures.

File system metadata layout for append-only storage

The techniques disclosed herein enable systems to efficiently interface with zoned namespace (ZNS) storage devices through a specialized management entity. To achieve this, the management entity receives write requests from a file system containing data intended for storage at the ZNS device. In response, the management entity selects a zone from the ZNS device to write the file data to. Accordingly, the file data is written by appending the file data to the zone at a location indicated by a write pointer. When the write operation is completed, the offset of the file data within the zone is observed and recorded by the file system in file metadata. In contrast to typical systems which allocate locations at the storage device prior to writing, appending file data and then recording the location enables improved efficiency in file system operations. Namely, that write operations can be issued to the ZNS device non-serially.

File system improvements for zoned storage device operations

A dataset is represented by an ordered set of B+ trees, with each snapshot of the dataset being represented by an immutable B+ tree. An ordering of B+ trees in the ordered set logically arranges those B+ trees into layers, with the mutable B+ tree representing the current state of the dataset being logically arranged as an upper layer, and with immutable B+ trees representing snapshots of the dataset being progressively arranged as lower layers with the oldest snapshot being a lowest layer. A snapshot deletion includes merging the snapshot's corresponding B+ tree with its highest neighbor B+ tree.

Managing snapshotting of a dataset using an ordered set of B+ trees

Methods, systems, and computer storage media for providing reduced-latency data operations for data or storage in file systems. The file system implements a set of policies that indicate how data operations are performed. In operation, a request for access to storage of a file system to perform a data operation is received. The file system supports both a bitmap representation and a range-only representation of the storage. A bitmap representation of a region for performing the data operation is communicated because, for certain data operations, the file system operates based on communicating only bitmap representations while restricting access to range-only representations. The range-only representations are a compacted representation of storage of the file system. Based on the bitmap representation of the region, the storage of the file system is accessed. The bitmap representation of the region further includes a bitmap lock that restricts access to storage corresponding to the bitmap representation.

Reduced-latency data operations for files in a file system

Scale out file system using refs and scale out volume

A file system is implemented in a system comprising an anonymous-write storage device. A request for a file system operation that comprises semantics of the file system to receiving the request is received. The file system translates the file system operation to one or more commands that effect the file system operation while maintaining consistency with operation of the anonymous-write storage device. The one or more commands comprise one or more of updating file system metadata, sending a request to the anonymous-write storage device, or receiving a response from the anonymous-write storage device. The anonymous-write storage device is configured to write data in an append-only format and determine locations of data write operations agnostic of input of a write location from the file system. Metadata is updated to indicate results of the file system operation.

File system for anonymous write

Methods, systems, and computer storage media for providing self-healing operations for root corruptions in a file system are provided. While performing file system operations, the root may be inaccessible by the file system engine for executing file system operations. A root becomes inaccessible for performing file system operations when corrupted. A determination is made that the root having a root data region is corrupted and a self-healing operation should be performed. A root-corruption self-healing operation is executed to re-allocate (or recycle) the root data region to the root. The self-healing operation circumvents a copy-on-write policy of the file system to re-allocate the root data region of the root in-place and transform the root from a corrupted root to a recycled root. A file system data operation, such as a data recovery operation or file system management operation (e.g., a file or directory operation) can be performed using the recycled root.

Self-healing operations for root corruption in a file system

A current state of a dataset is represented by a mutable B+ tree, while each snapshot of the dataset is represented by an immutable B+ tree. An ordering of B+ trees in the ordered set logically arranges those B+ trees into layers, with the mutable B+ tree representing the current state of the dataset being logically arranged as an upper layer, and with immutable B+ trees representing snapshots of the dataset being progressively arranged as lower layers with the oldest snapshot being a lowest layer. Each B+ tree stores dataset records with a strong or a weak reference state; strong records are independent of any lower-layer record, while weak records are dependent on at least one lower-layer record. A snapshot deletion includes merging the snapshot's corresponding B+ tree with its highest neighbor B+ tree, including merging overlapping records based on their weak vs. strong reference state.

Methods, systems, and computer storage media for providing data operations using hard links (hard link operations) for files in a file system are provided. Accessing files using hard link operations is based on File_Name-to-File_ID mappings and File_ID-to-File_Object mappings stored in hard link data structures. In operation, a file name for file content is received to perform a data operation. The file content is accessed using the file name. The file name is associated with a hard link data structure having a File_Name-to-File_ID mapping and a File_ID-to-File_Object mapping. The file name is also associated with an alternate file name for the file content. The alternate file name is associated with an alternate hard link data structure having an alternate File_Name-to-File mapping and the File_ID-to-File_Object mapping. The alternate file name is received. The file content is accessed using the alternate file name to perform an alternate data operation on the file content.

Hard link operations for files in a file system

Methods, systems, and computer storage media for providing reduced-latency data operations in a file system. In operation, a request to perform a data operation associated with a B+ table storage of a file system is received. The file system supports the B+ table storage that is accessible using a B+ tree lookup logic for accessing B+ table data and supports a cache storage that is accessible using a cache table lookup logic for accessing cache data in the cache storage. A number of references to the location in the B+ table storage is determined. The location in the B+ table storage is associated with the request to perform the data operation. Based on determining the number of references to the location, a cache storage location to perform the data operation is accessed. The cache storage location is mapped (using the cache table) to the location in the B+ table storage.

Data operations using a cache table in a file system

The use of steaming functionality on a storage device may be optimized by performing a combination of stream and non-stream writes based on a size of the data being written to a given stream. For example, a method may comprise writing data associated with a plurality of files to a first set of one or more erase blocks, determining that an amount of data associated with a given one of the plurality of files in the first set of one or more erase blocks has reached a threshold, and moving the data associated with the given file from the first set of one or more erase blocks to a stream, the stream comprising a second set of one or more erase blocks on the storage device different from the first set of one or more erase blocks.

Opportunistic use of streams for storing data on a solid state device

An append-only streams capability may be implemented that allows the host (e.g., the file system) to determine an optimal stream size based on the data to be stored in that stream. The storage device may expose to the host one or more characteristics of the available streams on the device, including but not limited to the maximum number of inactive and active streams on the device, the erase block size, the maximum number of erase blocks that can be written in parallel, and an optimal write size of the data. Using this information, the host may determine which particular stream offered by the device is best suited for the data to be stored.

Append only streams for storing data on a solid state device

Growing community of inventors

Kirkland, WA, United States of America

Rajsekhar Das