Hiddenlayer, Inc.

Zeliang Kan

Jason Martin

Ravikumar Balakrishnan

Hengrui Jia

A plurality of queries are input into an artificial intelligence (AI) model. The AI model is made up of a plurality of layers including an input layer, an output layer, and at least one intermediate layer between the input layer and the output layer. Each intermediate layer, during inference, can output a plurality of activations. Thereafter, for each query, activations are intercepted from at least one of the intermediate layers. It is then determined whether a distribution of the intercepted activations across the queries indicates that the queries seek to cause the AI model to behave in an undesired manner by conducting a distance-based similarity analysis between the intercepted activations and reference activations. Data characterizing such determination is then provided to a consuming application or process. Related apparatus, systems, techniques and articles are also described.

Growing community of inventors

London, United Kingdom

Zeliang Kan