International Business Machines Corporation

David Harold Berk

Uri Shani

Ariel Landau

Dagan Gilat

Sergey Krutyolkin

Joel Arditi

Aviad Sela

Inna Skarbovsky

Alex Akilov

Yaakov Dolgov

A data processing method is provided. The method comprises collecting a stream of data records received from one or more data sources connected in a communications network; dividing the stream of data records into sets of data records for parallel processing by a plurality of concurrently running tasks, wherein a first task loads a persistent index associated with a first set of data records into memory to generate an in-memory version of the first persistent index for the first set of data records; and identifying duplicate and non-duplicate data records in the first set of data records, based on searching the in-memory version of the first persistent index.

Duplicate filtering in a data processing environment

A data processing method is provided. The method comprises collecting a stream of data records from one or more devices in a network; loading one or more persistent indexes associated with the stream of data records into memory; identifying duplicate data records in the stream of data records using the in-memory indexes; and updating a repository such that the duplicate data records are not stored in the repository or managed differently than non-duplicate data records.

A relational database architecture with Dynamic Load capability, including a relational database for storing records, a storage unit, a record dispatcher for storing a plurality of objects, each containing at least one record, in the storage unit, and a mapping of the objects to the records which they contain, where the record dispatcher is configured to receive a Dynamic Load Request including a key value of desired records, search the mapping to determine which of the objects in the storage unit contain records that match the key value, and load any of the matching records into the database.

Relational database architecture with dynamic load capability

Growing community of inventors

Haifa, Israel

David Harold Berk