Security and privacy inspection of bidirectional generative artificial intelligence traffic using api notifications

Abstract

Disclosed is a cloud-based security system implemented using API notifications provided by a GenAI service or application. The security system provides bidirectional traffic inspection to protect against privacy and security concerns related to the GenAI services. The security system receives notifications of traffic including requests directed to the GenAI service from endpoints as well as the GenAI responses. The security system includes a GenAI request classifier trained to classify prompts as benign, prompt injection attack, or uploaded files. The security system further includes a GenAI response classifier trained to classify responses as normal, leaked system prompt, leaked user uploaded files, or leaked training data. Based on the classification, and optionally other security analysis, the security system may enforce security policies based on both the requests and responses that may include triggering alerts to administrators, deleting data stored by the GenAI service, and the like.