Intuit, Inc.

Rajendra Tiwari

Saikiran Sri Thunuguntla

Vishal Reddy Baddam

Suman Ghosh

Systems and methods are described for batch materialization of an incremental change data capture (CDC) changeset with full row changes. The primary keys are extracted from the incremental CDC changeset and an indication of the extracted primary keys are broadcast to a plurality of executors. The primary keys may be added to Bloom filter or a plurality of Bloom filters that are broadcast to the executors. Each executor filters a baseline data table based on the extracted primary keys to generate a baseline match dataframe with all primary keys matching the extracted primary keys, and a baseline unmatched dataframe with all primary keys not matching the extracted primary keys. Each executor receives full row changes from a partitioned incremental CDC changeset and combines the changes with the baseline unmatched dataframe to produce a final changed baseline data table.

Growing community of inventors

Bengaluru, India

Rajendra Tiwari