Hewlett-packard Company

Computer Associates Think, Inc.

David J Van Maren

Jeff J Kato

Bruce A Thompson

David W Ruska

Matthew C Dewey

James M Wilson

Jerry A Blades

Virgil Kay Russon

Mark John Simms

Jeffrey Dicorpo

Peter Joseph Bramhall

John G McCarthy

R Andrew Purcell

Michael Rottinghaus

Mass storage system controller apparatus receives requests for access to files in a data storage system and manages access to the files in the data storage system, the data storage system having a plurality of storage devices and the requests being addressed to the data storage system as a single virtual storage device having addressable Command Regions allocated to controller commands and addressable Mapped Data Regions allocated for data storage. The apparatus includes a file system to manage access to files in the data storage system, the file system having a processor acting as a Command Region processor and acting as a Mapped Data Region processor. A disk emulator responds as a disk drive device to write/read commands and passes the write/read commands addressed to Command Regions of the virtual storage device to the Command Region processor, the write/read commands addressed to Command Regions containing controller commands for the file system. The Command Region processor responds to controller commands in the write/read commands and processes the controller commands to control files in the data storage system. The disk emulator responds as a disk drive device to write/read commands and passes write/read commands addressed to a Mapped Data Region of the virtual storage device to the Mapped Data Region processor. The Mapped Data Region processor responds to a write/read command with a Mapped Data Region address and transfers data to or from the data storage system for files mapped to the Mapped Data Region address.

Intelligent controller accessed through addressable virtual space

A method and apparatus for storing user data organized into a plurality of data records on tape is provided. User data is first accepted from a host computer. The user data is then organized into data groups independent of the record structure. Records are then assigned to data entities where each data entity has at least one data record. A group index is formed to which information pertaining to the records and/or entities contained within the group is written. Data is then stored on the tape in groups along with each associated group index.

Storing plural data records on tape in an entity with an index entry

A method for storing a set of files on a multiple volume media set supports the ISO/IEC 13346 standard for optical media while minimizing swapping and temporary storage requirements when the multiple volume media set is used in an optical disk autochanger. The method allows the media set (e.g., optical disks) to be self-describing in accordance with the ISO/IEC 13346 standard. However, when the media set is on-line, the meta-data is separated from the data to optimize on-line performance. The method sequentializes the meta-data of the file set before writing it to the media set to achieve these advantages.

Method for storing data files on a multiple volume media set

A Transparent Peripheral File System (TPFS) includes a Peripheral File System Adapter (PFSA) which communicates with a host operating system at the vnode level of file operation by packaging such communications for transmission over an I/O system interface, such as SCSI. A file peripheral system remote from the host and in a peripheral relation thereto is responsive to the PFSA, without an intervening file server. The peripheral file system produces hardware commands for the mass storage device whose space it manages. The peripheral file system may be embedded in a mass storage device, a lump in the interconnecting interface cable, or a smart interface card in the backplane of the host. The peripheral file system may include a daisy chain connection to allow the propagation of vnode communication to other peripheral file systems. In this way a hierarchy of peripheral file systems may be physically mounted to one another in a way that corresponds to how they are logically mounted. A Character To File Translator (CFX) allows a TPFS to emulate raw mode access, even if the host system is not equipped with a vnode file operation interface. CFX does this by converting to and from streams of characters communicated with an application using this ersatz raw mode, while instead of storing the data on the storage medium itself, actually relying upon the remote peripheral file system for storing, or having stored, the streams of characters as a file.

Transparent peripheral file systems with on-board compression,

The present invention provides a method and apparatus for compressing user data and for storing the user data on magnetic tape. The user data is accepted and organized into a plurality of records. The user data is then compressed according to a compression algorithm involving converting at least some of the user data to codewords using a dictionary which is derived from the data. The compressed data is flushed from memory before a new dictionary is created. The user data may be written to magnetic tape after being organized into groups, the groups being independent of the record organization. A codeword indicating the start of a new dictionary is inserted at the beginning of a group and preferably at the beginning of the first record within a group. Records may be further organized into entities. A codeword indicating a flush operation is inserted following each entity and a codeword indicating the start of a new dictionary is inserted at the beginning of the first entity within a group.

Data compression method and apparatus utilizing an adaptive dictionary

Storage of compressed data with algorithm

A tape packet assembler/disassembler (TPAD) for a tape drive accumulates host-transmitted records, which may or may not be blocked by the host. The accumulated host-transmitted records are combined into packets comprising a data portion made up of data blocks and of a trailer portion made up of table entries describing how to disassemble the packet. Host-transmitted commands to write a tape mark are intercepted by the TPAD and replaced by corresponding information in the trailer. Packets are accumulated in a buffer in the TPAD, and are written to the tape drive when the buffer becomes full, or upon a command from the host to reposition the tape, such as rewind. The interblock gaps that would ordinarily separate the host-transmitted records now separate packets, resulting in a considerable increase in the efficiency of tape utilization. The TPAD maintains in the trailer various pointers to assist in forward and back spacing by file and records. It also can automatically and adaptively compress and decompress the user's data. To assist in this it also maintains in the trailer information identifying data blocks containing a dictionary reset.

High capacity tape drive transparently writes and reads large packets of

A tape packet assembler/disassembler (TPAD) for a tape drive accumulates host-transmitted records, which may or may not be blocked by the host. The interblock gaps that would ordinarily separate the host-transmitted records now separate packets, resulting in a considerable increase in the efficiency of tape utilization. When the user attempts to modify an existing collection of application records the entire packet containing that collection must first be read to place its content into the TPAD, the revision created in the buffer, and then a revised version of the entire packet written back to the tape. A failure in this process can result in the loss or corruption of data in that part of the packet preceding the revision. This loss can come as a surprise and be unacceptable to the user. To prevent such corruption or loss the TPAD does not attempt to overwrite the original version of the packet receiving the modification until it has successfully written a demonstration packet (equal in content to the revised packet) a safe distance down the tape beyond the packet to be overwritten. Only then is an attempt made to perform the actual overwrite on the original packet. If it succeeds the demonstration packet is ignored and may eventually be overwritten by subsequent activity. If the overwrite of the original packet fails for any reason, it is simply erased, creating a large (and harmless) interblock gap ahead of the demonstration packet, which now serves as the permanent replacement version.

Magnetic tape packet assembler/disassembler safeguards existing data

A tape packet assembler/disassembler (TPAD) for a tape drive accumulates host-transmitted records, which may or may not be blocked by the host. The accumulated host-transmitted records are combined into packets comprising a data portion made up of data blocks and a trailer portion made up of table entries describing how to disassemble the packet. Host-transmitted commands to write a tape mark are intercepted by the TPAD and replaced by corresponding information in the trailer. Packets are accumulated in a buffer in the TPAD, and are written to the tape drive when the buffer becomes full, or upon a command from the host to reposition the tape, such as rewind. The interblock gaps that would ordinarily separate the host-transmitted records now separate packets, resulting in a considerable increase in the efficiency of tape utilization. The TPAD maintains in the trailer various pointers to assist in forward and back spacing by file and records. It also can automatically and adaptively compress and decompress the user's data. To assist in this it also maintains in the trailer information identifying data blocks containing a dictionary reset.

A data compression system implementing expansion protection employs one or more pairs of FIFOs to compare the lengths of raw and processed versions of a block of received data. The shorter version is transmitted so that the data transmitted by the data compression system is at most negligibly expanded relative to the system input. A code is inserted in the output stream to indicate the beginning of the transmission of a raw data block so that a receiving or retrieving system can determine whether the data following needs to be decompressed or not. Further codes can be injected to indicate a switch from raw data to processed data in the output of the compression system.

Data compression system with expansion protection

An adaptive data compression system is reset when performance drops below a predetermined threshold to permit greater compression of long files with evolving distributions of symbol combinations. The compression system uses a resettable dictionary in which initially unassigned codes are strategically assigned to symbol combinations as they are encountered in the data stream. The difference between the bit-lengths of corresponding lengths of the compressor input and output is compared with a value representing a predetermined performance threshold. The dictionary can be reset if the actual performance falls below the performance threshold. The reset can be inhibited while the dictionary is less than half-full to ensure that performance measures are statistically significant. However, if the performance is such that data expansion is occurring, reset is not so delayed.

Performance-based reset of data compression dictionary

A FIFO shift register (100) includes a parallel data in-port (PIN) to each of its cells (101-132) and a means for managing input to determine for each cell whether it is to receive data and, if so, whether through its conventional serial in-port (SIN) or through its parallel in-port. The input manager comprises a bidirectional shift register of input manager cells arranged in one-to-one correspondence with data cells. A one-bit validity indicator stored within a given input manager cell is logically combined with asserted PUSH and PULL signals to determine the source of data for the associated data cell and its immediate successor. This arrangement not only provides greater speed by minimizing bubble-through time, but permits the FIFO shift register to be clocked. This capacity for synchronous operation permits ready VLSI implementation with concomitant advantages in economy, reliability and speed.

Growing community of inventors

Fort Collins, CO, United States of America

David J Van Maren