Intel Corporation

Daniel Cutter

Vinodh Gopal

James D Guilford

Gilbert M Wolrich

Wajdi K Feghali

Matthew James Adiletta

Kirk S Yap

Debra Bernstein

William Ralph Wheeler

Christopher Dolan

William C Hasenplaugh

Jose Niell

Gunnar Gaubatz

James Redfield

Ravishankar Iyer

Blaise B Fanning

Ioannis T Schoinas

Ramadass Nagarajan

Deepak Limaye

Robert P Ottavi

Erdinc Ozturk

Pierre Laurent

James D Allen

Stephen J Robinson

Ravi Kolagotla

Simon N Peffers

Mukesh Patel

Quang T Le

Gregory B Tucker

Shadi T Khasawneh

George Powley

John J Vranich

Andrew Milne

An integrated circuit includes a compression accelerator to process a request from software to compress source data into an output file. The compression accelerator includes early-abort circuitry to provide for early abort of compression operations. In particular, the compression accelerator uses a predetermined sample size to compute an estimated size for a portion of the output file. The sample size specifies how much of the source data is to be analyzed before computing the estimated size. The compression accelerator also determines whether the estimated size reflects an acceptable amount of compression, based on a predetermined early-abort threshold. The compression accelerator aborts the request if the estimated size does not reflect the acceptable amount of compression. The compression accelerator may complete the request if the estimated size reflects the acceptable amount of compression. Other embodiments are described and claimed.

Technology for early abort of compression acceleration

Methods and apparatus relating to verifying a compressed stream fused with copy or transform operation(s) are described. In an embodiment, compression logic circuitry compresses input data and stores the compressed data in a temporary buffer. The compression logic circuitry determines a first checksum value corresponding to the compressed data stored in the temporary buffer. Decompression logic circuitry performs a decompress-verify operation and a copy operation. The decompress-verify operation decompresses the compressed data stored in the temporary buffer to determine a second checksum value corresponding to the decompressed data from the temporary buffer. The copy operation transfers the compressed data from the temporary buffer to a destination buffer in response to a match between the first checksum value and the second checksum value. Other embodiments are also disclosed and claimed.

Verifying compressed stream fused with copy or transform operations

An embodiment of an integrated circuit may comprise a hardware compressor to compress data, the hardware compressor including circuitry to store input data in a history buffer, compute one or more code tables based on the input data, and compute a compression stream header based on the computed one or more code tables. Other embodiments are disclosed and claimed.

Flexible compression header and code generation

Apparatus and method for efficient compression block decoding using content-addressable structure for header processing. For example, one embodiment of an apparatus comprises: a header parser to extract a sequence of tokens and corresponding length values from a header of a compression block, the tokens and corresponding length values associated with a type of compression used to compress a payload of the compression block; and a content-addressable data structure builder to construct a content-addressable data structure based on the tokens and length values, the content-addressable data structure builder to write an entry in the content-addressable data structure comprising a length value and a count value, the count value indicating a number of times the length value was previously written to an entry in the content-addressable data structure.

Method and apparatus for efficient deflate decompression using content-addressable data structures

A lossless data compressor of an aspect includes a first lossless data compressor circuitry coupled to receive input data. The first lossless data compressor circuitry is to apply a first lossless data compression approach to compress the input data to generate intermediate compressed data. The apparatus also includes a second lossless data compressor circuitry coupled with the first lossless data compressor circuitry to receive the intermediate compressed data. The second lossless data compressor circuitry is to apply a second lossless data compression approach to compress at least some of the intermediate compressed data to generate compressed data. The second lossless data compression approach different than the first lossless data compression approach. Lossless data decompressors are also disclosed, as are methods of lossless data compression and decompression.

Apparatus and method for two-stage lossless data compression, and two-stage lossless data decompression

A processing device comprising compression circuitry to: determine a compression configuration to compress source data; generate a checksum of the source data in an uncompressed state; compress the source data into at least one block based on the compression configuration, wherein the at least one block comprises: a plurality of sub-blocks, wherein the plurality of sub-block includes a predetermined size; a block header corresponding to the plurality of sub-blocks; and decompression circuitry coupled to the compression circuitry, wherein the decompression circuitry to: while not outputting a decompressed data stream of the source data: generate index information corresponding to the plurality of sub-blocks; in response to generating the index information, generate a checksum of the compressed source data associated with the plurality of sub-blocks; and determine whether the checksum of the source data in the uncompressed format matches the checksum of the compressed source data.

Supporting random access of compressed data

An apparatus and method for performing efficient lossless compression. For example, one embodiment of an apparatus comprises: first compression circuitry to identify and replace one or more repeated bit strings from an input data stream with distances to the one or more repeated bit strings, the first compression circuitry to generate a first compressed data stream comprising literal-length data identifying a first instance of each repeated bit string and distance data comprising distances from the first instance to each repeated instance of the repeated bit string; second compression circuitry to perform sorting, tree generation, and length calculations for literal-length values and distance values of the first compressed data stream, the second compression circuitry comprising: variable length code mapping circuitry to map each literal-length value and distance value to a variable length code; header generation circuitry to generate a header for a final compressed bit stream using the length calculations; and a transcoder to substitute the variable length codes in place of the literal-length and distance values to generate a compressed bit stream body, wherein the transcoder operates in parallel with the header generation circuitry; and bit stream merge circuitry to combine the header with the compressed bit stream body to generate a final lossless compressed bitstream.

Method and apparatus for high performance compression and decompression

Technologies for database acceleration include a computing device having a database accelerator. The database accelerator performs a decompress operation on one or more compressed elements of a compressed database to generate one or more decompressed elements. After decompression of the compressed elements, the database accelerator prepares the one or more decompressed elements to generate one or more prepared elements to be processed by an accelerated filter. The database accelerator then performs the accelerated filter on the one or more prepared elements to generate one or more output elements. Other embodiments are described and claimed.

Technologies for big data analytics accelerator

In an example, there is disclosed an apparatus, comprising: a data store comprising a hash table having for at least some rows a hash entry indexed by a hash value, and comprising a hash chain of one or more pointers to a history buffer, and a spill counter; and one or more logic elements, including at least one hardware logic element, comprising a data compressor to: inspect a string0 comprising n bytes at position p in a data file; get the spill counter from a hash entry corresponding to string0; inspect a string1 comprising n bytes at p+k, wherein k is a positive integer; get the spill counter from a hash entry corresponding to string1; determine that the spill counter for string1 is less than the spill counter for string0; and search a chain1 (the hash chain of a hash entry corresponding to string1) for a matching string of size at least n+k with an offset of &#x2212;k.

Optimized selection of hash collision chains

An apparatus is described. The apparatus includes a main memory controller having a point-to-point link interface to couple to a point-to-point link. The point-to-point link is to transport system memory traffic between said main memory controller and a main memory. The main memory controller includes at least one of compression logic circuitry to compress write information prior to being transmitted over the link; decompression logic circuitry to decompress read information after being received from the link.

System memory having point-to-point link that transports compressed traffic

In an embodiment, a processor comprises a plurality of processing cores and a compression accelerator to compress an input stream comprising a first data block and a second data block. The compression accelerator comprises a first compression engine to compress the first data block; and a second compression engine to update state data for the second compression engine using a sub-portion of the first data block; and after an update of the state data for the second compression engine using the sub-portion of the first data block, compress a second data block using the updated state data for the second compression engine. Other embodiments are described and claimed.

Managing state data in a compression accelerator

Methods and apparatus are described by which data is compressed using semi-dynamic Huffman code generation. Embodiments generate symbol statistics over a portion of data. The symbol statistics are expanded to include all possible literals that could appear within the data. Any literal or reference added to the statistics may be given a frequency of one. The statistics are used to generate a semi-dynamic Huffman code. The entire data is then compressed using the semi-dynamic Huffman code.

Semi-dynamic, low latency compression

A compute device to generate deterministic compressed streams receives a current string to be matched to one or more prior instances of the current string, the current string being located within an input buffer and the one or more prior instances located within a history buffer. The compute device identifies a limited subset of index memory designated for storing pointers to the prior instances, identifying a reserved slop region in the index memory, and compares the current string to a prior instance, locating the at least one prior instance using at least one pointer to the at least one prior instance. The at least one pointer is stored within the limited subset of the index memory, and the compute device also prohibits use of any pointers stored in the reserved slop region of the index memory. Other embodiments are described and claimed.

Growing community of inventors

Maynard, MA, United States of America

Daniel Cutter