Dell Products Inc.

Raul Gracia

Vinicius Michel Gottin

Rômulo Teixeira De Abreu Pinho

Joel Evans Christner

Disclosed systems and methods employ a data prefetch mechanism for tiered streaming storage systems. Disclosed features improve performance and efficiency of batch reads, i.e., read operations that require data imported from a long-term storage service. In at least some embodiments, disclosed systems and methods execute asynchronous, parallel prefetch reads from long-term storage while a reader is performing batch reads. The prefetched data may correspond to the next data required by the reader and, if so, the reader can perform the next read operation without incurring the latency associated with a synchronous read from long term storage. Disclosed systems and methods thereby beneficially improve throughput for streaming batch readers.

Prefetch mechanism for tiered streaming storage systems

A method includes identifying, by a server module running at a stream data platform, a set of data to be moved to a target data storage tier of the stream data platform, querying, by the server module, service level agreement (SLA) objectives of the target data storage tier, fetching, by the server module, the set of data, selecting, by the server module, a compressor based on the SLA objectives, compressing the set of data using the selected compressor, and storing the set of data in the target data storage tier.

Multi-objective compression for data tiering in a stream data platform

One example method includes receiving stream data batches at a client node. The client node determines if the stream data batches are compressible. For the compressible stream data batches, a request is sent to a server node for an on-demand stream compression service, the request including an indicator of a stream data type for the compressible data batches. The on-demand stream compression service is deployed and launched at the client node. The on-demand stream compression service includes a compressor pool of compressors that are able to compress the stream data type of the compressible stream data batches. A compressor of the compressor pool is selected and used to compress the compressible stream data batches. The compressed stream data batches are sent to the server node.

Growing community of inventors

Barcelona, Spain

Raul Gracia