Amazon Technologies, Inc.

Jimit Majmudar

Sachin Purushottam Joglekar

Rahul Gupta

Ben Smith

Lili Gehorsam

Devices and techniques are generally described for security threat mitigation for generative machine learning models. In some examples, first request data including a first request may be received. First prompt data may be generated based at least in part on the first request data. First plan data may be generated, the first plan data including a first API call to a first API of a first computer-implemented service. The first API call may be executed and first result data may be received in response to the first API call. A determination may be made that the first result data includes a first impermissible instruction to inject data into a subsequent prompt. First output data may be generated indicating that the first request cannot be completed.

Security threat mitigation for large language models

Devices and techniques are generally described for security threat mitigation for generative machine learning models. In some examples, first prompt data including first data associated with a first natural language input and a first span may be determined. An LLM may determine first plan data using the first prompt data. The first plan data may include a call to the first API. A first classifier model may determine a first trust score for the first span. A first attention score may be determined for the first span and the first action plan. Second plan data may be generated based on at least one of the first trust score and the first attention score or the second trust score and the second attention score.

Growing community of inventors

Sunnyvale, CA, United States of America

Jimit Majmudar