Trifacta Inc.

Wei Zheng

Sean Philip Kandel

Philip John Vander Broek

Adam Silberstein

Edward Eli Marschner

Alon Bartur

Zain Asgar

A system standardizes values that occur in large datasets before the dataset is analyzed. The system identifies values in a dataset that are similar to each other and associates those values with each other to form groups. The system determines a canonical value for each group of associated values. Within each group, the system replaces values that have been associated with each other with the canonical value for the group. As a result, the dataset is transformed into a dataset that has standardized values, and the standardized dataset is provided as input for analysis by a data analysis system. By standardizing the dataset in this manner, the data analysis system can process a larger portion of the dataset.

Standardizing values of a dataset

A system determines samples of datasets that are typically processed by big data analysis systems. The samples are for use for development and testing of transformations for preprocessing the datasets in preparation for analysis by big data systems. The system receives one or more transform operations input datasets for the transform operations. The system determines samples associated with the transform operations. According to a sampling strategy, the system determines samples that return at least a threshold number of records in the result set obtained by applying a transformation. According to another sampling strategy, the system receives criteria describing the result of the transform operations and determines sample sets that generate result sets satisfying the criteria as a result of applying the transform operations.

Sampling for preprocessing big data based on features of transformation results

Transformation based sampling for preprocessing big data

Growing community of inventors

San Carlos, CA, United States of America

Wei Zheng