Microsoft Technology Licensing, LLC

Microsoft Corporation

Brian D Andrew

Luis Felipe Cabrera

Thomas Jay Miller

Norbert Paul Kusters

Surendra Verma

Mark Joseph Zbikowski

Gary D Kimura

Jacob Lacouture

Praerit Garg

Jonathan M Cargille

Jianrong Gu

Keith Stuart Kaplan

Vishal Vikram Ghotge

Sameet Harishanker Agarwal

Klaus U Schutz

David Golds

Peter T Brundrett

James W Kelly

Robert P Reichel

Benjamin A Leis

Daniel Edward Lovinger

Wesley Allan Witt

Jon Cargille

Daniel W H Chan

Supriya Wickrematillake

Mike K Lai

Vishal A Ghotge

Thomas J Miller

A file system enables the real-time correction of detected corruptions to on-disk data. An enhancement to a file system responds in real time to file system corruptions detected on a running volume, and repairs the corruptions at the point where the file system detects them. Upon detection of a corruption by the file system, the system enhancement records information describing the nature of the corruption. A repair scan is defined for each type of corruption encountered. Repair scans can be run at the top level of execution in the current thread in which the corruption is detected, or they may require a dedicated thread to service the repair operation.

Real-time file system repairs

A system and method supporting multiple independent transactional resource managers on a single logical volume, in which each resource manager is a unit of storage management. Each resource manager independently maintains metadata associated with a collection of files that are contained within the scope of a resource manager, such as the files within a subdirectory corresponding to the resource manager. A general architecture allows databases or other entities to integrate with a transactional file system volume via the resource managers, whereby each database has its files independently associated with a transactional resource manager such that database operations do not affect other operations on the volume. Having independent resource managers allows different levels of performance, reliability, feature availability, and manageability to be specified within a single volume. Functions to create, start, and shut down a resource manager are also provided, along with other functions that facilitate resource manager usage.

Method and system for supporting multiple independent transactional resource managers on a single logical volume

An external request to perform a file operation within a resource manager is received. The resource manager selects a resource manager file identifier that is to be an attribute of a file associated with the file operation. The file operation and the resource manager file identifier are logged in a log entry of a log. Subsequent to logging the log entry, the log entry is accessed during a forward pass through the log, for example, during a roll-forward recovery. If it is determined that the resource manager file identifier in the log entry is assigned to the file, the first log entry is not processed. On the other hand, if it is determined that the resource manager file identifier in the log entry is not assigned to the file, the resource manager file identifier is selected to be assigned to a file when the first log entry is processed.

Identifying files within a namespace

The location of appropriate undo data necessary for reversing an effect of an operation performed at a computer system is identified during a forward pass through a log. A cycle count can be used to determine if appropriate undo data is stored in an object or is stored in auxiliary storage. If appropriate undo data is stored in an object and the undo data would be potentially lost by processing a redo log entry to modify the contents of the object, the appropriate undo data is copied to auxiliary storage. In some embodiments, a secondary computer system receives log entries from a primary computer system. The secondary computer system can identify undo data necessary to reverse an effect of an operation performed at the secondary computer system from received log entries. Identified undo data can be copied auxiliary storage at the secondary computer system.

Identifying appropriate undo during a forward pass through a log

A system and method for encryption and decryption of files. The system and method operate in conjunction with the file system to transparently encrypt and decrypt files in using a public key-private key pair encryption scheme. When a user puts a file in an encrypted directory or encrypts a file, data writes to the disk for that file are encrypted with a random file encryption key generated from a random number and encrypted with the public key of a user and the public key of at least one recovery agent. The encrypted key information is stored with the file, whereby the user or a recovery agent can decrypt the file data using a private key. With a correct private key, encrypted reads are decrypted transparently by the file system and returned to the user. One or more selectable encryption and decryption algorithms may be provided via interchangeable cryptographic modules.

Encrypting file system and method

A system and method for more efficiently maintaining multiple temporal snapshots of a common base volume. When the base volume is modified, such as when existing data is overwritten with new data, that modification may affect two or more of the snapshots. Before the modification, the existing data is copied only to the differential file associated with the latest snapshot. When a region of a selected snapshot is read, the region is read from the selected snapshot's associated differential file. If the existing data is in the differential file, that data is returned to the reading process. If later snapshots have been taken, data associated with other parts of the region may be stored in one or more of the later differential files. If the differential file of the selected snapshot does not have data for each portion of the requested region, the mechanism continues by accessing each differential file associated with subsequent snapshots in temporal order from the earliest following the selected snapshot to the latest until either the region is complete or no later snapshots remain. Finally, if any part of the region was not filled in with data from one of the differential files, then that part of the region is read from the base volume.

Support for multiple temporal snapshots of same volume

A snapshot system capable of capturing snapshots of multiple volumes wherein the snapshots are coordinated. A snapshot manager determines which volumes are to be involved in a snapshot operation, and issues a message to the file system for each volume involved, each message including information sufficient to identify the volumes involved in the snapshot operation. Each file system passes its respective message down to a coordinator mechanism. The coordinator mechanism coordinates the state of each of the volumes, such as by holding writes thereto, to put each volume into a quiescent state, and then enables the snapshot to be captured. When the snapshots are captured, a snapshot set will include snapshots that are coordinated across the multiple volumes. The coordinator mechanism releases any writes being held for the volumes involved.

Kernel-based crash-consistency coordinator

A system and method for creating a snapshot with a differential file maintained on the base volume that can grow as needed. When a snapshot is captured, free space is allocated on the base volume to receive the differential file. Writes to the base volume are allowed except to the free space allocated to the differential file. Then the snapshot is captured. After the snapshot process is complete, data that was originally present at the time the snapshot was captured may be copied to the differential file before it is modified. To grow the differential file out of its allocated space, new free space is selected from the free space currently on the base volume in conjunction with the free space at the time the snapshot was captured. The free space bitmap file of the snapshot volume may be used to identify the free space at the time the snapshot was captured.

System and method for growing differential file on a base volume of a snapshot

A system and method for encryption and decryption of files. The system and method operate in conjunction with the file system to transparently encrypt and decrypt files in using a public key-private key pair encryption scheme. When a user puts a file in an encrypted directory or encrypts a file, all data writes to the disk for that file are encrypted with a random file encryption key generated from a random number and encrypted with the public key of a user and the public key of at least one recovery agent. The encrypted key information is stored with the file, whereby the user or a recovery agent can decrypt the file data using the private key thereof. When a proper private key is used, encrypted reads from the disk are decrypted transparently by the file system and returned to the user.

A change journal for recording changes to files in a storage volume of a computer system keeps a record for each notable change to a file. Each record is given a unique update sequence number, which is a serial number of ever-increasing value assigned to each record. Each record includes one or more change reasons specifying what type of action occurred with respect to the associated file. The presence of a close file change reason in a record connotes that the record includes all the notable changes made to the file in a preceding file session, which is defined as the time occurring after the last time a close file change reason was entered in a change record, up to the time of the next succeeding close file change reason. For each file, an entry is made in the storage volume's master file table referencing the change record that currently includes the most up-to-date information on the status of the change reasons for the file.

Journaling ordered changes in a storage volume

In order to decrease the overall cost of storing large amounts of data, systems have been developed that use a hierarchy of storage devices from fast local disks to archival off-line storage. Such storage devices may be managed in a hierarchy where data that is accessed only infrequently can be moved to archival storage. The present invention relies on a tight integration of a hierarchical storage manager into the I/O system so that remotely stored attributes can be identified and tracked internally to the I/O system just like any other attributes. Implementations of the present invention may rely on a layered driver model where lower level drivers detect the existence of files with remotely stored attributes and then transfer control for processing I/O requests involving files with remotely stored attributes to higher level drivers. The higher level drivers then assume control to finish processing the I/O request.

Growing community of inventors

Redmond, WA, United States of America

Brian D Andrew