System and method for obfuscation by common function and common function prototype

Abstract

Disclosed herein are systems, methods, and computer-readable storage media for obfuscating by a common function. A system configured to practice the method identifies a set of functions in source code, generates a transformed set of functions by transforming each function of the set of functions to accept a uniform set of arguments and return a uniform type, and merges the transformed set of functions into a single recursive function. The single recursive function can allocate memory in the heap. The stack can contain a pointer to the allocated memory in the heap. The single recursive function can include instructions for creating and explicitly managing a virtual stack in the heap. The virtual stack can emulate what would happen to the real stack if one of the set of functions was called. The system can further compile the source code including the single recursive function.