Liveramp, Inc.

Acxiom Corporation

Adam Zimmerman

Pavan Roy Marupally

Dwayne Collins

Joseph Shannon Duncan

Terry Talley

Derek Leonard

Terry Michael Talley

A method for improving fault tolerance of MapReduce computing systems processing extremely large data sets employs a custom mapper script that outputs the results to the shuffle and reduce processes. The Reducer writes its results into multiple files that are then compressed. The resulting files are processed by a map-only Reducer job that aggregates and reduces the results of a single file into a single output. The non-block compression prevents the file from being split across multiple mappers. The job can recover if a single mapper fails without requiring that each Map node be rerun, thus improving fault tolerance.

Fault tolerant big data processing

A system and method for efficiently matching into an extremely large data set uses two data stores. A search data store contains identifiers that correspond to localized portions of the main data store, along with matching values for that portion of the main data store. To perform a matching operation, the search data store is first searched for a matching value. Then the identifier for the localized portion of the main data store is retrieved from the search data store. Using this identifier, a search may be performed against only the corresponding localized portion of the main data store rather than the entire main data store.

System and method for matching into a complex data set

An entity resolution data structure system compares two data graphs by creating a confusion matrix in a distributed processing environment. A benchmark file is created from one data graph for comparison to a reference data graph. Identifiers and metadata are appended to the benchmark file to allow the comparison to take place and the construction of a confusion matrix. The confusion matrix provides a high-level indication of the results of the comparison. When the data graphs contain personally identifiable information (Pll), the process does not require any Pll to be transmitted in either direction between the parties who maintain the data graphs to be compared.

Entity resolution data structure system and method

A system and method for transforming input data in a data graph is structured in such a way that it does not destroy embedded contextual data yet also keeps the number of edges in the data graph sufficiently small in number that computation with respect to the data in the data graph is feasible with existing computational resources on extremely large graph sets. Incoming data is represented as a collection of 'cliques' rather than placing each data object into its own node in the graph database. Maintaining the clique structure though the graph build pipeline dramatically reduces the exponential increase in the number of edges in the graph, while also maintaining all of the contextual data presented on the input record.

Asserted relationship data structure

An apparatus and method to increase accuracy in individual attributes derived from anonymous aggregate data uses aggregation keys in order to retrieve distribution sets and generate best-effort results for individual attributes. Multiple aggregation keys may be utilized to which individual attributes may be cross-mapped. The aggregation keys may be divided into location-based aggregation keys and name-based aggregation keys. The resulting data may be of varying granularity depending upon the granularity of the aggregation key used for the distribution and to generate the attributes.

Growing community of inventors

Conway, AR, United States of America

Adam Zimmerman