Machine learning for constrained mutation-based fuzz testing

Abstract

Techniques for constrained mutation-based fuzzing are described. Machine accesses an input file of code for testing. Machine performs multiple runs of a fuzzing algorithm using the input file and the code. Each run includes: performing a mutation of one or more bytes of the input file and determining which parts of the code were executed when the code was run with the mutated input file. Machine stores, for each run, an indication of whether the mutation caused execution of a portion of the code which was not executed prior to the mutation, Machine generates heatmap of the input file based on the stored indications. The heatmap maps each of the bytes in the input file to a value indicating whether the mutation of the byte caused execution of the portion of the code for testing which was not executed prior to the mutation. Machine tailors fuzzing algorithm based on heatmap.