Intuit, Inc.

Itsik Yizbak Mantin

Ron Bitton

A method includes receiving, at a server from a user device, a user query to a large language model (LLM), creating an LLM query from the user query and an application context, gathering confidential information from the LLM query, and sending the LLM query to the LLM. The method includes receiving, from the LLM, an LLM response to the LLM query, comparing the LLM response to the confidential information to generate comparison result, and setting a leakage detection signal based on comparison result.

Leakage detection for large language models

A method includes receiving, at a server from a user device, a user query to a large language model (LLM), creating an LLM query from the user query, inserting a system prohibited request into the LLM query to generate a revised LLM query, and sending the revised LLM query to the LLM. The method further includes receiving, from the LLM, a first LLM response to the LLM query, testing the first LLM response to detect whether a prohibited response to the system prohibited request is included in the first LLM response, and setting a prompt injection signal based on whether the prohibited response to the system prohibited request is included in the first LLM response.

Growing community of inventors

Hod HaSharon, Israel

Itsik Yizbak Mantin