Alibaba Group Holding Limited

Shaomeng Wang

Xu Yang

Xiaoyan Jiang

Bin Dai

Shen Li

Wei Chu

Yuan Qi

Zihao Fu

Ning Cai

Embodiments of the present disclosure provide a method and system for training a model by using training data. The training data includes a plurality of samples, each sample includes N features, and features in the plurality of samples form N feature columns, and the method includes: determining an importance value of each of the N feature columns; determining whether the importance value of each of the N feature columns satisfies a threshold condition; performing a dimension reduction on M feature columns to generate P feature columns in response to the determination that the importance values of the M feature columns do not satisfy the threshold condition, wherein M&#x3c;N and P&#x3c;M; merging (N&#x2212;M) feature columns having importance values that satisfy the threshold condition and the generated P feature columns to obtain (N&#x2212;M+P) feature columns; and training the model based on the training data including the (N&#x2212;M+P) feature columns.

Method and system for training model by using training data

Methods and apparatuses for training model based on random forest are provided. The method includes: dividing worker nodes into one or more groups; performing random sampling, by worker nodes in each group, in the preset sample data to obtain the target sample data; and training, by the worker nodes in each group, one or more decision tree objects using the target sample data. Example embodiments of the present disclosure do not need to scan the complete sample data for once, thereby greatly reducing the amount of data to be read, the time cost, and further the iterative update time of the model. The efficiency of training is improved.

Method and apparatus for training model based on random forest

The present disclosure provides feature data processing methods and devices. One exemplary feature data processing method comprises: classifying features into an important feature set and an auxiliary feature set according to information attribute values of the features; converting features in the auxiliary feature set to hash features; and combining the hash features with features in the important feature set, and setting the combined features as fingerprint features. Training and prediction of to-be-processed data can be performed based on the fingerprint features. With the embodiments of the present disclosure, training dimensions can be more controllable and the amount training data amount can be reduced. Therefore, the efficiency of data processing can be improved.

Feature data processing method and device

There is provided an association analysis method and apparatus. An original database is divided into projection databases, each not contributing to a support count of a frequent item set of another. The projection databases are used for sequential-pattern association analysis performed respectively by nodes corresponding to the projection databases. Local frequent item sets and corresponding support counts obtained by the nodes are combined. Since an established projection database does not contribute a support count of a frequent item set of another projection database, different nodes can perform association mining, including pruning, on different projection databases respectively.

Growing community of inventors

Hangzhou, China

Shaomeng Wang