Detecting and preventing execution of software exploits

Abstract

In non-limiting examples, anti-exploit systems and methods described herein monitor a memory space of a process for execution of functions. Stack walk processing is executed upon invocation of one of the functions in the monitored memory space. During execution of the stack walk processing, at least one memory check is performed to detect suspicious behavior. An alert of suspicious behavior is triggered when the performing of the memory check detects at least one of: code execution attempted from non-executable memory, identification of an invalid base pointer, identification of an invalid stack return address, attempted execution of a return-oriented programming technique, the base pointer is outside a current thread stack, and a return address is detected as being inside a virtual memory area. If an alert of suspicious behavior is triggered, execution of a payload is prevented for the invoked function.