Network security for machine learning models including large language models (llms)

Abstract

There is provided a system for securing an AI-application guided by a LLM, comprising: a processor executing a code for: receiving a session of an interaction by an entity with the LLM-guided AI-application, the session including a plurality of output tokens generated by the AI-application in response to a plurality of input tokens inputted into the AI-application by the entity and/or from at least one external resource, dynamically during processing of the session by the LLM, tracing control and/or data-flow within attention-heads of transformer layers of the LLM with respect to the session, to identify at least one output token of the plurality of output tokens logically stemming from at least one input token of the plurality of input tokens originating from the at least one external resource, and triggering a security action for preventing the at least one external resource from feeding adversarial input into the AI-application.