Data mining technique with shadow individuals

Abstract

Roughly described, a computer-implemented evolutionary data mining system includes a memory storing a candidate gene database containing active and shadow individuals; a gene pool processor which tests only active individuals on training data and updates their fitness estimates; a competition module which selects individuals (both active and shadow) for discarding from the gene pool in dependence upon both their updated fitness estimate and their testing experience level; and a gene harvesting module providing for deployment selected ones of the individuals from the gene pool. The gene database has an experience layered elitist pool, and individuals to compete only with other individuals in their same layer. Shadow individuals are created in each layer for active individuals that survive all competition with the layer before their testing experience exceeds the testing experience range for the layer.