Method and apparatus for providing and handling traps

Abstract

A computer program or a computer process is provided by replacing a native computer instruction with a trapping computer instruction which is the size of the native computer instruction and which, when executed, causes a trap to the kernel. A trap handler in the kernel determines that the inserted trapping computer instruction caused the trap and transfers control to a user trap handler. The user trap handler maps the trap site to a patch of computer instructions. When the trapping computer instruction is executed, the trap handler transfers control from the kernel to the user trap handler which in turn transfers control to the patch. Native computer instructions in sufficient proximity to corresponding patches of computer instructions may be replaced with branching computer instructions of the size of the native computer instruction and which transfer control to those corresponding patches. Other native computer instructions are replaced with trapping computer instructions which transfer control to corresponding patches through traps to the kernel. Unanticipated traps may be handled by mapping all trap sites other than those corresponding to inserted trapping computer instructions to a default patch which processes unanticipated traps. Native computer instructions may be replaced with a selected privileged computer instruction. The kernel trap handler transfers control to the user trap handler if a trap is caused by the selected privileged computer instruction and a user trap handler is installed. Information regarding affected components of the state of the computer process prior to the trap is retrieved and stored.