Ab Initio Technology LLC

Ab Initio Software Corporation

Ephraim Meriwether Vishniac

Stephen J Schmidt

Craig W Stanfill

Vrishal Kulkarni

Paul Bay

Marshall Alan Isman

John L Richardson

H Mark Bromley

Joseph Skeffington Wholey, Iii

Igor Sherb

Joseph Skeffington Wholey

Joseph Skeffington Iii Wholey

A system for managing storage of data units includes a data storage system configured to store multiple data blocks, at least some of the data blocks containing multiple data units, with at least a group of the data blocks being stored contiguously, thereby supporting a first read operation that retrieves data units from at least two adjacent data blocks in the group. The system is configured to perform two or more operations with respect to data units. The operations include: a second read operation, different from the first read operation, that retrieves a data unit to be read based at least in part on an address of a data block containing the data unit to be read, and a delete operation that replaces a first data block containing a data unit to be deleted with a second data block that does not contain the deleted data unit.

Managing operations on stored data units

A system for managing storage of data units includes a data storage system configured to store multiple data blocks, at least some of the data blocks containing multiple data units, and configured to store, for at least some of the data blocks, corresponding historical information about prior removal of one or more data units from that data block, the removal affecting at least some addresses of data units in that data block. The system is configured to perform at least one operation that accesses at least a first data unit stored in a first data block according to address information interpreted based on any stored historical information corresponding to the first data block.

A system for managing storage of data units includes a data storage system configured to store multiple data blocks, at least some of the data blocks containing multiple data units, with at least a group of the data blocks being stored contiguously, thereby supporting a first read operation that retrieves data units from at least two adjacent data blocks in the group. The system is configured to perform one or more operations with respect to data units, the operations including a delete operation that replaces a first data block containing a data unit to be deleted with a second data block that does not contain the deleted data unit, with the second data block having the same size as the first data block.

Managing data by: receiving a group of individually accessible data units, each data unit identified by a key value, with key values determined such that the key value identifying a first data unit received before a second data unit occurs earlier in a sort order than the key value identifying the second data unit; and processing the data units for storage in a data storage system. The processing includes: storing blocks of data, the blocks being generated by combining a plurality of the data units; providing an index with entries that enable location, based on a provided key value, of a block that includes a data unit corresponding to the provided key value; and generating one or more screening data structures associated with the blocks for determining, based on a given key value, whether to search the stored blocks for a data unit corresponding to the given key value.

Managing storage of individually accessible data units

A method for managing data includes receiving individually accessible data units, each identified by a key value; storing a plurality of blocks of data, each of at least some of the blocks being generated by combining a plurality of the data units; and providing an index that includes an entry for each of the blocks. One or more of the entries enable location, based on a provided key value, of a block that includes data units corresponding to a range of key values that includes the provided key value.

Managing data includes: receiving at least one group of individually accessible data units over an input device or port, each data unit identified by a key value, with key values of the received data units being sorted such that the key value identifying a given first data unit that is received before a given second data unit occurs earlier in a sort order than the key value identifying the given second data unit; and processing the data units for storage in a data storage system. The processing includes: storing a plurality of blocks of data, each of one or more of the blocks being generated by combining a plurality of the data units; providing an index that includes an entry for each of the blocks, wherein one or more of the entries enable location, based on a provided key value, of a block that includes data units corresponding to a range of key values that includes the provided key value; and generating one or more screening data structures associated with the stored blocks for determining a possibility that a data unit that includes a given key value was included in the group of individually accessible data units.

Executing graph-based computations includes: accepting a specification of a computation graph in which data processing elements are joined by linking elements; dividing the data processing elements into sets, at least one of the sets including multiple of the data processing elements; assigning to each set a different computing resource; and processing data according to the computation graph, including performing computations corresponding to the data processing elements using the assigned computing resources.

Managing computing resources in graph-based computations

A data processing system and method that provides checkpointing and permits a continuous flow of data processing by allowing each process to return to operation after checkpointing, independently of the time required by other processes to checkpoint their state. Checkpointing in accordance with the invention makes use of a command message from a checkpoint processor that sequentially propagates through a process stage from data sources through processes to data sinks, triggering each process to checkpoint its state and then pass on a checkpointing message to connected &ldquo;downstream&rdquo; processes. This approach provides checkpointing and permits a continuous flow of data processing by allowing each process to return to normal operation after checkpointing, independently of the time required by other processes to checkpoint their state.

Growing community of inventors

Arlington, MA, United States of America

Ephraim Meriwether Vishniac