Hiddenlayer, Inc.

James Ballard

Tanner Burns

Chris Sestito

Eoin Wickens

Thomas Bonner

Marta Janus

A machine learning model is scanned to detect actual or potential threats. The threats can be detected before execution of the machine learning model or during an isolated execution environment. The threat detection may include performing a machine learning file format check, vulnerability check, tamper check, and stenography check. The machine learning model may also be monitored in an isolated environment during an execution or runtime session. After performing a scan, the system can generate a signature based on actual, potential, or absence of detected threats.

Scanning and detecting threats in machine learning models

Adversarial attacks on a machine learning model are detected by receiving vectorized data input into the machine learning model along with outputs of the machine learning model responsive to the vectorized data. The vectorized data corresponds to a plurality of queries of the machine learning model by a requesting user. A confidence level is determined which characterizes a likelihood of the vectorized data being part of a malicious act directed to the machine learning model by the requesting user. Data providing the determined confidence levels can be provided to a consuming application or process. Multi-tenant architectures are also provided in which multiple machine learning models associated with different customers can be centrally monitored.

Machine learning model adversarial attack monitoring

A system detects and responds to malicious acts directed towards machine learning models. Data fed into and output by a machine learning model is collected by a sensor. The data fed into the model includes vectorization data, which is generated from raw data provided from a requester, such as for example a stream of timeseries data. The output data may include a prediction or other output generated by the machine learning model in response to receiving the vectorization data. The vectorization data and machine learning model output data are processed to determine whether the machine learning model is being subject to a malicious act (e.g., attack). The output of the processing may indicate an attack score. A response for handling the request by a requester may be selected based on the output that includes the attack score, and the response may be applied to the requestor.

Growing community of inventors

Fredericksburg, VA, United States of America

James Ballard