Branch on cache hit/miss for compiler-assisted miss delay tolerance

Abstract

In a computer system having a hierarchical memory, the problem of tolerating cache miss latency is solved by dynamically switching appropriately between two different code sequences, one optimized at compile-time, assuming a cache-hit, and the other optimized at compile-time, assuming a cache-miss. A method for processing instructions and data in a computer system including a hierarchical memory and a static instruction sequence including a memory access instruction and associated memory access latency specific code sequences, each code sequence optimized dependent on an execution of the memory access instruction causing one of a hit or a miss at a level of the memory hierarchy, includes the steps of: decoding and executing the memory access instruction and storing information indicating whether the execution of the memory access instruction caused the hit or the miss; and branching to a cache hit optimized code sequence when the information indicates the hit and a miss optimized code sequence when the information indicates the miss, responsive to the step of storing. Preferably, the memory access latency specific code sequences are associated with one or more identified critical miss-points. The step of branching may be responsive to an inserted branch instruction associated with the memory access instruction. The branch instruction may also specify a level of the cache memory upon which the step of branching is recommended.