Microsoft Technology Licensing, LLC

Aaron William Ogus

Bradley Gene Calder

Cheng Huang

Huseyin Simitci

Ju Wang

Jin Li

Andrew James Edwards

Richard John Black

Antony Ian Taylor Rowstron

Austin N Donnelly

Yikang Xu

Evan J Schrier

Serguei Anatolievitch Legtchenko

Douglas Wayne Phillips

Samuel James McKelvie

Shane Kumar Mainali

Vadim Makhervaks

Ioan Oltean

Sergey M Yekhanin

Jason David Adrian

Hugh Williams

Yu Lin Chen

Thomas R Reardon

Madhav Himanshubhai Pandya

Jinyang Li

Shuai Mu

Douglas Phillips

Mark Edward Shaw

Andreas Georgiou

Jacob Rubin Lorch

Scott Chao-Chueh Lee

Bryan Stephen Matthew

Ioan Alexandru Stefanovici

James Hilton Clegg

David Lara Saucedo

Sridhar Srinivasan

Hugh Evan Williams

Garret Buban

Reinhard Sebastian Bernhard Nowozin

Benn Charles Thomsen

Parikshit Santhan Gopalan

Alexander Lloyd Gaunt

Rokas Drevinskas

Ariel Gomez Diaz

Patrick Neil Anderson

Hugh David Paul Williams

Andromachi Chatzieleftheriou

John Christopher Dainty

Hakon Gabriel Verespej

Michael J Narayan

Raluca Andreea Diaconu

Mengyang Yang

Rajsekhar Das

Maneesh Sah

Roderick Michael Toll

Arild Einar Skjolsvold

Krishnan Varadarajan

Daniel S Glasser

George Joy

Jegan Devaraju

Paul E Newson

Rushi Srinivas Surla

Haiyong Wang

Laura Marie Caulfield

Omar Carey

Lee Edward Prewitt

Manish Chablani

Zhong Deng

Ramesh Shankar

Marcus Swenson

Muhammad Junaid Shahid

Peng Huang

Pradeep Seela

Christopher John Tector

Jieqing Wang

Girish Saini

James Mason Boswell

Jianfeng Zhu

Srinivasan Malayala

Weixiang Sun

Mauricio David Zaragoza Ibarra

Arlid E Skjolsvold

Yiking Xu

David E Glerok

Parikshit Santhana Gopalan

Tejas Shah

Zhenshan Yu

Terry Chen

Yilong Zhao

Yiheng Tao

Subhasish Bhattacharya

Abhishek Singh Baghel

Ameya Bhagat

Ping-Yao Tseng

Fnu Arunachaleshwar

Richard Black

Austin Donnelly

Sergui Anatolievitch Legtchenko

A server system is provided that includes one or more compute nodes that include at least one processor and a host memory device. The server system further includes a plurality of solid-state drive (SSD) devices, a local non-volatile memory express virtualization (LNV) device, and a non-transparent (NT) switch for a peripheral component interconnect express (PCIe) bus that interconnects the plurality of SSD devices and the LNV device to the at least one processor of each compute node. The LNV device is configured to virtualize hardware resources of the plurality of SSD devices. The plurality of SSD devices are configured to directly access data buffers of the host memory device. The NT switch is configured to hide the plurality of SSD devices such that the plurality of SSD devices are not visible to the at least one processor of each compute node.

Local non-volatile memory express virtualization device

A server system comprising processing devices, disks and a storage fabric, all arranged to operate according to a storage fabric protocol in order to communicate between the processing devices and the storage devices over the storage fabric, wherein the storage fabric is settable with a mapping determining which of the processing devices are allocated to use which of the storage devices in providing said services to the client systems. The system comprises a control function arranged to dynamically adapt the mapping in response to either: (i) failure or other downtime of one of the processing devices, storage devices, a component of the storage fabric or a connection in the storage fabric; or (ii) a current demand for the services from the client systems.

Server system

Various embodiments, methods, and systems for erasure coding with overlapped local reconstruction codes, are provided. An erasure coding scheme can be defined based on Overlapped Local Reconstruction Codes (OLRC) that achieve high storage efficiency by providing fault tolerance properties that optimize reconstruction for common cases of failures while maintaining the reconstruction costs for uncommon case of failures. In operation, a data chunk is divided into data fragments. The data fragments correspond to zones. A plurality of parity fragments is computed using the data fragments. A parity fragment is computed using a subset of the data fragments. The plurality of parity fragments are assigned to the zones comprising the data fragments, where the data fragments and the plurality of parity fragments define overlapped local construction codes having a plurality of local groups. An unavailable data fragment is recoverable from at least two local groups from the plurality of local groups.

Erasure coding with overlapped local reconstruction codes

Described herein is a system and method for utilizing a protocol over RDMA network fabric between a first computing node and a second computing node. The protocol identifies a first threshold and a second threshold. A transfer request is received, and, a data size associated with the transfer request is determined. Based up the data size associated with the transfer request, one of at least three transfer modes is selected to perform the transfer request in accordance with the first threshold and the second threshold. Each transfer mode utilizes flow control and at least one RDMA operation. The selected transfer mode is utilized to perform the transfer request.

Transport protocol and interface for efficient data transfer over RDMA fabric

In various embodiments, methods and systems for implementing a distributed metadata management system in distributed storage systems are provided. A distributed storage system operates based on data storage resources (e.g., extents and streams). The distributed metadata management system is implemented for extent and stream metadata to facilitate the scalability of metadata processing. The distributed storage system implements extent managers and stream managers that independently manage extent and stream metadata, respectively. The extent managers are associated with an extent table that stores extent metadata. The stream managers are associated with streams that store associations with extents. The distributed metadata management system can also utilize a bootstrap layer that leverages components of a legacy distributed storage system to facilitate distributed management of extent and stream metadata. The bootstrap layer is used to store the extent table as a system table and to persist the state of the stream manager as system streams.

Distributed metadata management in a distributed storage system

In various embodiments, methods and systems for implementing distributed data object management are provided. The distributed data object management system includes a distributed storage system having a local metadata-consensus information store in and one or more remote metadata-consensus information stores. A metadata-consensus information store is configured to store metadata-consensus information. The metadata-consensus information corresponds to erasure coded fragments of a data object and instruct on how to manage the erasure coded fragments. The distributed storage system further includes a local data store and one or more remote data stores for the erasure coded fragments. The distributed data object management system includes a distributed data object manager for operations including, interface operations, configuration operations, write operations, read operations, delete operations, garbage collection operations and failure recovery operations. The distributed data object management system is operates based on metadata paths and data paths, operating in parallel, for write operations and read operations.

Distributed data object management system

A server system comprising storage devices, processing devices and a storage fabric all operating according to a storage fabric protocol. The storage fabric comprises a plurality of individual switches having a modular design from which an overall switch is built, and the individual switches have individual respective configuration settings which determine which processing devices are allocated to use which of the storage devices. The system comprises an API enabling a software control function to configure the overall switch. The API is operable to receive from the control function an overall mapping of the storage devices to the processing devices instead of requiring the individual configuration settings of each of the individual switches to be specified by the control function, the API being configured to convert the overall mapping into the individual configuration settings of the individual switches to produce the overall mapping.

In various embodiments, methods and systems for implementing distributed data object management are provided. The distributed data object management system includes a local metadata-consensus information store and one or more remote metadata-consensus information stores for metadata-consensus information and a local data store and one or more remote data stores for erasure coded fragments. For a write operation, corresponding metadata writes and data writes are performed in parallel using a metadata write path and a data write path, respectively, when writing to the local metadata-consensus information store and the one or more remote metadata-consensus information stores and the local data store and the one or more remote data stores. And, for a read operation, corresponding metadata reads and data reads are performed in parallel using a metadata read path and a data read path, respectively, when reading from the metadata-consensus information stores and the data stores.

Distributed data object management system operations

The techniques described herein are configured to improve the remanufacturing process by waiting to rebuild selective data items stored on a storage device that is unavailable. A storage device is unavailable when it is taken offline and/or disconnected from a network. The storage device may be taken offline due to a failed component (e.g., an actuator arm, an actuator head, damage to the underlying storage media, etc.). The storage device comprises multiple independent logical units, where a logical unit is a uniquely addressable portion of a storage device that is visible and/or that is exposed to a host. Accordingly, the techniques described herein are configured to conserve resources by selectively rebuilding data items stored in logical units of an offline storage device that are not affected by a remanufacturing process implemented due to a failed component.

Maintenance of storage devices with multiple logical units

Techniques for implementing zone redundant computing services in distributed computing systems are described herein. In one embodiment, a computing device is configured to query a database in a first availability zone for a list of computing services corresponding to a user account and provided by servers in a second availability zone different than the first availability zone. The computing device can then selects one or more computing services provided by the servers in the second availability zone from the queried list of computing services and transmit an instruction to the servers in the second availability zone for performing the requested data operation.

Zone redundant computing services using multiple local services in distributed computing systems

In various embodiments, methods and systems for implementing garbage collection in distributed storage systems are provided. The distributed storage system operates based on independent management of metadata of extent and stream data storage resources. A hybrid garbage collection system based on reference counting garbage collection operations and mark-and-sweep garbage collection operations is implemented. An extent lifetime table that tracks reference weights and mark sequences for extents is initialized and updated based on indications from extent managers and stream managers, respectively. Upon determining that an extent is to be handed-off from weighted reference counting garbage collection operations to mark-and-sweep garbage collection operations, a reference weight field for the extent is voided and a mark sequence field of the extent is updated. The mark sequence field is updated with a latest global sequence number. The mark-and-sweep garbage collection operations are utilized to reclaim the extent when the extent is no longer referenced.

Hybrid garbage collection in a distributed storage system

A data-storage system comprises a head receiver configured to variably receive up to a number M of write heads. The data-storage system also includes an installed number N of write heads arranged in the head receiver, a substrate receiver configured to receive one or more data-storage substrates, and a positioner machine configured to adjust a relative placement of each of the M write heads with respect to at least one of the one or more data-storage substrates.

Disaggregated data storage and retrieval system

Growing community of inventors

Redmond, WA, United States of America

Aaron William Ogus