Utilizing a multi-layer security architecture for agent reinforcement fine-tuning of a machine learning model

Abstract

The present disclosure relates to systems, non-transitory computer-readable media, and methods for utilizing a reinforcement fine-tuning pipeline to fine tune an agent machine learning (ML) model utilizing user-defined grader endpoints and/or external tools with a multi-layer security architecture. For example, the disclosed systems can utilize a multi-layer security architecture to filter incoming training datasets, perform content refusal checks, chain-of-thought leak detections, and/or governance oversights prior to training the agent ML model, during active training of the agent ML model using external tools and/or grader models, and/or during post-training of the agent ML model (prior to releasing a fine-tuned snapshot of the model). In addition, the disclosed systems can generate stateful trajectory rollouts to associate training trajectories of the agent machine learning model to unique identifiers to facilitate multiple environments and/or tasks to run concurrently while maintaining consistency across external tool calls and grading interactions during training.