System and method for efficiently generating association rules using scaled lift threshold values to subsume association rules

Abstract

A data processing system processes data sets (such as low-resolution transaction data) into high-resolution data sets by mapping generic information into attribute-based specific information that may be processed to identify frequent sets therein. When association rules are generated from such frequent sets, the complexity and/or quantity of such rules may be managed by removing redundancies from the rules, such as by filtering subsumed rules from the generated rule set that have a confidence metric value that does not exceed a first confidence metric value for a subsuming rule by more than a scaled lift threshold value that is calculated by determining a complement of the first confidence metric value, squaring the complement to obtain a squared value and multiplying the squared value by a scaling factor.