Emc Corporation

Jianwang Ao

Caleb E Welton

Yi Yang Ren

Guangxin Yang

A method, article of manufacture, and apparatus for process data. In some embodiments, this includes receiving a desired sample size from a user, increasing the desired sample size based on a policy, using the increased desired sample size to determine a number of samples to take from a database segment, wherein the database segment includes a portion of a dataset, generating a sub-dataset based on the determined number of samples to take from the database segment, wherein the sub-dataset only includes identifiers, joining the generated sub-dataset with the dataset, and storing the joined dataset in a storage device.

Policy based over sampling with replacement

A method, system, and process for representing a decision tree in a tabular format is discussed. The format may contain all the necessary information to traverse the nodes in parallel on a distributed system while consuming an efficient amount of resources. In some embodiments, the tree may be stored in a relational database as a table.

Decision tree representation for big data

A method, article of manufacture, and apparatus for process data. In some embodiments, this includes determining a dataset, wherein the dataset includes a plurality of data values and a plurality of identifiers, and wherein the plurality of identifiers are in a non-continuous order, and wherein the dataset is stored across a number of segments, determining a number of samples to take from the dataset, identifying a minimum identifier and a maximum identifier, and a total number of identifiers, calculating a density based on the plurality of identifiers, minimum identifier and maximum identifier, calculating a number of samples to take from a segment based on the calculated density, the number of samples to take from the dataset, and the number of segments, generating a sub-dataset based on the calculated number of samples to take from a segment, joining the generated sub-dataset with the dataset, and storing the joined dataset in a storage device.

Efficient sampling with replacement

A method, system, and process for generating attribute class statistics for a decision tree is discussed. In an embodiment the ACS's are calculated in parallel on a distributed architecture, and may consume a constant amount of resources.

Generating attribute-class-statistics for decision trees

A method, system, and process for classifying a dataset using a decision tree is discussed. The decision tree may be stored in a novel format on a distributed architecture, and the dataset classification may be performed in parallel on the distributed architecture.

Growing community of inventors

Beijing, China

Jianwang Ao