Sun Microsystems, Inc.

Mitsubishi Electric Information Technology Center America, Inc.

Oracle America, Inc.

Mitsubishi Electric Research Laboratories, Inc.

Mitsubishi Electric Information Technology Center America, Inc (ita)

John H Howard

Randy B Osborne

David G Robinson

Qin Zheng

Hugh C Lauer

Randall David Rettberg

Chia Shen

Ross T Casley

Abhijit Ghosh

Christopher Stein

Douglas J Hahn

Harufusa Kondoh

An object-based storage may employ a transactional interface and a copy on write protocol. The inode for the file (which is identified by the I-number and indicates one or more blocks storing the file data) may be copied to a working copy of the inode, and the working copy may be updated to indicate the newly allocated blocks. In response to a file commit (e.g. a file close or a file synchronization command), the working copy of the inode may be atomically written to a non-volatile storage. In this manner, the updates performed during the transaction may be atomically committed to the file. The inode itself may be stored in a file (referred to as the inode file), and the storage may perform updates to the inode file atomically as well. The inode file may be modified using the copy on write protocol described above, and the master inode (mapping the blocks of the inode file) may be atomically written to non-volatile storage to atomically update the inode file. In one particular embodiment, the storage employs a journal technique for updates to the inode file.

Object-based storage device with improved reliability and fast crash recovery

A system and method for synchronizing mirrored and striped disk writes. A data storage system may include a client computer system coupled to a first data storage device and a second data storage device and configured to transmit a first data write request. The first storage device may be configured to transmit a sequence number to the client computer system in response to receiving the first data write request. The client computer system may be further configured to transmit a second data write request including the sequence number to the second storage device. The second data storage device may include a counter and is configured to compare a current counter value to the sequence number. If the counter value is equal to the sequence number, the second storage device stores the data bytes corresponding to the second data write request and increments its counter.

System and method for synchronizing mirrored and striped disk writes

An apparatus includes a computing node and a metadata server. The computing node may transmit a request to open a file to the metadata server, which may provide a corresponding file identifier to the computing node. The computing node may use the file identifier to directly access the storage storing the file. In one embodiment, the storage may be an object-based storage. The storage may receive the file identifier with an access command and may perform the mapping from file identifier to storage blocks internally. Thus, accesses to the storage may be performed on an arbitrary boundary within the file. In other words, the entire block including the data needed by the computing node may not be transferred to the computing node. Instead, the storage may select the data requested from the file (e.g. via an offset and a number of bytes or some similar mechanism) and return that data to the computing node.

Direct access from client to storage device

A data storage system includes a computer coupled to a non-volatile storage, such as a disk drive. The computer includes a block cache for storing cached copies of data blocks, and a hash table that stores hash values corresponding to the data blocks. Prior to writing back a modified cache block to the non-volatile storage, a log recorder of the computer stores an updated hash value corresponding to the modified cache block within a write-ahead hash log, which is also contained in non-volatile storage. The log recorder creates a log record including an updated hash value and an address corresponding to a modified cache block. The log recorder additionally maintains a first pointer value indicative of log records that have been stored to the write-ahead hash log, and a second pointer value indicative of the most recent log record stored in the write-ahead hash log for which a corresponding modified cache block has been stored to the non-volatile storage. These pointer values are stored in the write ahead hash log with the log records. Log records are grouped into log blocks which are eventually written to the non-volatile storage as a group. After the log record containing the updated hash value has been successfully written to write-ahead hash log, a cache manager initiates the write-back of the dirty data block to the non-volatile storage.

Data storage system and method employing a write-ahead hash log

A file reconciliation process in a distributed file system uses a set of rnal or log files to track the history of file modification at each of different sites, or sets of directories, in a computer system. During reconciliation, sequences of version entries associated with each file in each journal are updated and compared to determine whether (1) a conflict exists for any of the files involved in the reconciliation, and (2) if not, which version of the file is the current version. The version entries contain a hash code or digest that to a high probability uniquely identifies the contents of a file. Sequences of hash codes are used to identify the sequence of file versions. Masks and site indicator fields are included in the journal files and used to track which journal files have copies of version entries for the purpose of deleting version entries when they become obsolete.

File version reconciliation using hash codes

A packet based data transmission system includes a flexible optimized nonocking transmit interface that incorporates optimized buffer modes, dynamic and static chaining, streaming and the utilization of small packet formats. Static chaining refers to connecting together the linked list for successive packets for the same transmit channel or virtual channel. Dynamic chaining refers to means by which the network interface performs this chaining automatically, thereby solving a blocking problem. On the transmit side, streaming refers to initiating the transmission of packet data before the entire packet data has been presented to the interface. This, in turn, permits more rapid recycling of the buffer space. On the receive side, streaming refers to initiating the processing of packet data before the entire packet has been received. The packet transmission system also includes a receive interface that incorporates a chunking system in which a buffer is divided into multiple chunks or segments to accommodate different size packets. Additionally, the receive interface includes an optimized linked list scheme to support chunking in which no linking element is required for the first buffer in the linked list. In one embodiment a small packet format is provided to reduce the relative overhead in sending small packets. In another embodiment the optimized buffer mode associated with the receive side can be utilized on the transmit interface for further reducing overhead.

Network interface

In an ATM network interface controller, a traffic management system is provided to allow implementation of available bit rate, or ABR, flow control by an external processor and the use of a new ABR controller within the traffic management system which provides functions that enable an external processor to control the flow control behavior of the network interface controller. The ABR controller accesses an external memory to control the flow of cells injected into the network. In one embodiment, the subject system uses a traffic shaper to control cell transmission rates, submits requests to an external memory which are processed by an external processor, and reads data from an external memory to adjust cell transmission rates and/or generates Resource Management, or RM, cells. More particularly, in one embodiment, the traffic shaper uses a timing chain approach enhanced with a delay queue which is capable of delaying transmission of i cell for a long period of time, thus being able to accurately control a wide range of cell transmission rates for multiple cell streams independently. The subject system submits requests to an external memory when it receives an RM cell, sends an RM cell, or when reaching a cell transmission limit, initiates processing by an external processor. An external processor communicates with the subject system by writing data into an external memory. The subject system reads data from the memory and determines what cell to send, how long it should be delayed before sending a next cell, thus allowing an external processor to easily control the behavior of an ATM network interface controller to implement a wide variety of flow control schemes.

Traffic shaping and ABR flow control

A switch for digital communication networks includes a queuing system cape of implementing a broad class of scheduling algorithms for many different applications and purposes, with the queuing system including both a tag-based primary queue which contain ATM cells organized by priority and a secondary queue which contains ATM cells which are not yet scheduled for transmission and which are organized by virtual channel. A queuing decision module is provided to determine in which queue an incoming ATM cell should be deposited. A requeuing module operates when an event occurs that unblocks a particular virtual channel. The requeuing module, on occurrence of such an event, accesses the secondary queue to obtain another cell, to assign it priority and to move it to the primary queue. The queuing decision module, along with a virtual channel table, can be used easily to block virtual channels when necessary. The combination of queues also allows for round robin scheduling.

ATM local area network switch with dual queues

In a distributed file environment, a system for safely updating a file wiut risk of losing work performed at one site due to work performed on the file at another site uses a journal or log at each site which is updated after a file is modified. This log is compared with the logs from other sites before a file is used at any one site, so that new versions can be propogated automatically and safely to out-of-date sites, with the user immediately alerted if conflicting versions of the file exist at different sites. The reconciliation can be applied to collections of files, automatically updating only those files for which it is safe to and necessary do so. Since reconciliation occurs at times selected by the user, inconsistent or partially completed versions of files need not be propogated to other sites. Additionally, logs may be built incrementally by occasionally observing the state of the systems in terms of the files and their time stamps and creating additional log entries reflecting appearance, disappearance and changes of files. Furthermore, logs may be purged of obsolete entries by including additional log entries indicating the most recent time each site has participated in a reconciliation and deleting obsolete entries that all sites have seen.

Method and apparatus for reconciling different versions of a file

A system for controlling traffic in a digital communication network to avoid data loss due to congestion utilizes both credit-based and rate-based traffic control approaches within a common framework, and adjusts the rate at which data is transmitted from a source in accordance with feedback in a form of credit values from a destination reflecting the ability of the network to transmit data and the destination to receive data. In one embodiment, the destination receives the information about network congestion, and, knowing its own capacity to receive more data, generates a credit in the form a numeric value which is fed back to the source so that the source adjusts its transmission rate to avoid data loss due to congestion in the network.

Growing community of inventors

Cambridge, MA, United States of America

John H Howard