Detecting self-modifying code in a pipelined processor with branch

Abstract

A pipelined x86 processor implements a method of detecting self-modifying code in which a prefetched block of instruction bytes may contain an instruction that is modified by a store instruction preceding it in the execution pipeline. The processor includes a Prefetch unit having a multi-block prefetch buffer, a Branch unit with a branch target cache (BTC), and a Load/Store (LDST) unit having store reservation stations. Self-modifying code is detected in three ways: (a) the Prefetch unit snoops store addresses from the LDST unit which are compared with (i) an address tag for each of the prefetch blocks of instruction bytes already loaded into the prefetch buffer, and (ii) the addresses of any pending prefetch requests, (b) the LDST unit snoops prefetch addresses issued by the Prefetch unit and compares them to store addresses queued in the store reservation stations, and (c) to ensure compatibility with the 486 specification for self-modifying code (which requires that a store that modifies an instruction be followed immediately by a jump to that instruction), the LDST unit detects when a store is followed by a COF that hits in the BTC which output a target address that is the same as the preceding store address. In particular, Prefetch unit snooping and LDST unit snooping detect instances of self-modifying code conditions that do not follow the 486 specification.