Autonomous simulation and synthetic response system for agentic ai systems

Abstract

A computer-implemented method, system, and non-transitory computer-readable medium for simulation-based testing and benchmarking of agentic artificial intelligence systems. A simulation agent is bound to at least one tool-access interface of an agentic AI system to replace at least one external tool. The simulation agent intercepts requests emitted at the tool-access interface, generates protocol-compliant responses using synthetic data and a simulated environment, and substitutes those responses for expected responses from the external tool, enabling task execution without invoking actual external tools. The agentic AI system executes one or more runs of a predefined task suite, including healthy and fault-injected runs, and actual agent outputs are evaluated against expected outputs to generate a performance vector. A simulation registry maintains metadata, schema definitions, and execution logs. The simulation agent selects among language model configurations based on reward scores and supports stealth observation mode to train surrogate tool models for closed-loop simulation.