Apparatus, system and method for an adaptive or static machine-learning classifier using prediction by partial matching (ppm) language modeling

Abstract

We have invented a process and method for creating a general-purpose adaptive or static machine-learning classifier using prediction by partial matching (PPM) language modeling. This classifier can incorporate homogeneous or heterogeneous feature types; variable-size contexts; sequential or non-sequential features. Features are ordered (linearized) by information saliency; and truncation of least-informative context is used for backoff to handle previously unseen events. Labels may be endogenous (from within the group) or exogenous (outside the group) of the feature types. Classification may generate labels and their probabilities; or only labels. Classification stores may be complete or minimized where redundant states are removed producing significant space savings and performance improvements. Classifiers may be static (unchanging) or online (adaptive or updatable incrementally or in batch). PPM classifiers may be incorporated in ensembles of other PPM classifiers or different machine learning algorithms. Training and prediction algorithms are both simple and efficient; and permit multiple implementations using standard software data structures. These benefits are achieved while providing state-of-the-art prediction performance.