Netezza Corporation

Ibm International Group Bv

Sharon Miller

Foster D Hinshaw

David L Meyers

William K Stewart

Michael Sporer

Raymond A Andraka

Barry M Zane

John Yates

Raymond J Andraka

An approach for providing compression of a database table that uses a compiled table algorithm (CTA) that provides leverage. Data within any given column in adjacent rows is often the same as or closely related to its neighbors. Rather than storing data in each column of each row as a specific integer, floating point, or character data value, a field reconstruction instruction is stored that when executed by a decompression engine can reconstruct the data value. The field reconstruction instruction may be bit granular and may depend upon past history given that the data compression engine may preserve state as row data is streamed off a storage device.

Technique for compressing columns of data

A distributed data processing system executes database operations, and includes a first group of host processors and a second group of job processing units (JPUs). A software application for executing database operations executes in a distributed fashion with portions of the database application executing on at least one central database processor and other portions executing on the data storage processors. At least a portion of the database application is implemented within and/or coordinated by a communication process that is executing the communication protocol. The communication process implements a load balancing function in front of the host processors, the load balancing function directing individual transactions to selected processors so as to evenly distribute workload.

Field oriented pipeline architecture for a programmable data streaming processor

A programmable streaming data processor that can be programmed to recognize record and field structures of data received from a streaming data source such as a mass storage device. Being programmed with, for example, field information, the unit can locate record and field boundaries and employ logical arithmetic methods to compare fields with one another or with values otherwise supplied by general purpose processors to precisely determine which records are worth transferring to memory of the more general purpose distributed processors. The remaining records arrive and are discarded by the streaming data processor or are tagged with status bits to indicate to the more general purpose processor that they are to be ignored. In a preferred embodiment, the streaming data processor may analyze and discard records for several reasons. The first reason may be an analysis of contents of the field. Other reasons for record blocking may have to do with tagging records that are to be visible to particular users depending upon a series of concurrent transactions.

Intelligent storage device controller

A field oriented pipeline processor for a data engine that can be programmed to recognize record and field structures of data received from a source such as a mass storage device. The processor has an interface for receiving field-delineated data from a field parser. The field parser parses non-field delineated data from a streaming data source into field delineated data under instruction from an external processing unit. The pipeline processor receives a field delineated data stream and employs logical arithmetic methods to compare fields with one another, or with values otherwise supplied by general purpose processors to precisely determine which records are worth transferring to memory of the more general purpose distributed processors. The architecture of the data engine allows for the use of substitution tables, temporary registers, and a data string register to assist in the efficiency and accuracy of the data engine processing.

Growing community of inventors

Boxborough, MA, United States of America

Sharon Miller