System and method of determining memory ownership on cache line basis for detecting self-modifying code including code with looping instructions

Abstract

A system and method of determining memory ownership on a cache line basis for detecting self-modifying code including code with looping instructions. An ownership queue includes multiple entries for determining memory ownership on a cache line basis. An ownership index and a wrap bit are determined for each cache line in the ownership queue, which are provided with each instruction derived from the same cache line. When an instruction is issued for execution, the ownership index provided with the instruction is used to access the corresponding entry in the ownership queue. If the instruction and entry wrap bits do not match, then an overwrite of the cache line is detected. The instruction is marked to invoke a first exception, which is performed when the instruction is ready to retire. The first exception flushes the processor, prevents the instruction from being retired, and re-fetches the instruction to continue processing.