Adaptive statistical regression and classification of data strings, with

Abstract

A data string is a sequence of atomic units of data that represent information. In the context of computer data, examples of data strings include executable programs, data files, and boot records consisting of sequences of bytes, or text files consisting of sequences of bytes or characters. The invention solves the problem of automatically constructing a classifier of data strings, i.e., constructing a classifier which, given a string, determines which of two or more class labels should be assigned to it. From a set of (string, class-label) pairs, this invention provides an automated technique for extracting features of data strings that are relevant to the classification decision, and an automated technique for developing a classifier which uses those features to classify correctly the data strings in the original examples and, with high accuracy, classify correctly novel data strings not contained in the example set. The classifier is developed using 'adaptive' or 'learning' techniques from the domain of statistical regression and classification, such as, e.g., multi-layer neural networks. As an example, the technique can be applied to the task of distinguishing files or boot records that are infected by computer viruses from files or boot records that are not infected.