International Business Machines Corporation

Narus, Inc.

Jialong Zhang

Zhongshu Gu

Heqing Huang

Ian M Molloy

Marc Philippe Stoecklin

Jiyong Jang

Taesung Lee

Xiaokui Shu

Dimitrios Pendarakis

Teryl Paul Taylor

Frederico Araujo

Dong Su

Ting Wang

Josyula R Rao

Xin Hu

Dhilung Hang Kirat

Bryant R Chen

Benjamin J Edwards

Wilka Carvalho

Suresh N Chari

Sung-Ju Lee

Heiko Hary Ludwig

Sabyasachi Saha

Marc Stoecklin

Guofei Gu

Marc Phillipe Stoecklin

Bruno Nardelli

A method, apparatus and computer program product to protect a deep neural network (DNN) having a plurality of layers including one or more intermediate layers. In this approach, a training data set is received. During training of the DNN using the received training data set, a representation of activations associated with an intermediate layer is recorded. For at least one or more of the representations, a separate classifier (model) is trained. The classifiers, collectively, are used to train an outlier detection model. Following training, the outliner detection model is used to detect an adversarial input on the deep neural network. The outlier detection model generates a prediction, and an indicator whether a given input is the adversarial input. According to a further aspect, an action is taken to protect a deployed system associated with the DNN in response to detection of the adversary input.

Detecting adversary attacks on a deep neural network (DNN)

Using a deep learning inference system, respective similarities are measured for each of a set of intermediate representations to input information used as an input to the deep learning inference system. The deep learning inference system includes multiple layers, each layer producing one or more associated intermediate representations. Selection is made of a subset of the set of intermediate representations that are most similar to the input information. Using the selected subset of intermediate representations, a partitioning point is determined in the multiple layers used to partition the multiple layers into two partitions defined so that information leakage for the two partitions will meet a privacy parameter when a first of the two partitions is prevented from leaking information. The partitioning point is output for use in partitioning the multiple layers of the deep learning inference system into the two partitions.

System for measuring information leakage of deep learning models

Decoy data is generated from regular data. A deep neural network, which has been trained with the regular data, is trained with the decoy data. The trained deep neural network, responsive to a client request comprising input data, is operated on the input data. Post-processing is performed using at least an output of the operated trained deep neural network to determine whether the input data is regular data or decoy data. One or more actions are performed based on a result of the performed post-processing.

Detecting adversarial attacks through decoy training

Deep learning training service framework mechanisms are provided. The mechanisms receive encrypted training datasets for training a deep learning model, execute a FrontNet subnet model of the deep learning model in a trusted execution environment, and execute a BackNet subnet model of the deep learning model external to the trusted execution environment. The mechanisms decrypt, within the trusted execution environment, the encrypted training datasets and train the FrontNet subnet model and BackNet subnet model of the deep learning model based on the decrypted training datasets. The FrontNet subnet model is trained within the trusted execution environment and provides intermediate representations to the BackNet subnet model which is trained external to the trusted execution environment using the intermediate representations. The mechanisms release a trained deep learning model comprising a trained FrontNet subnet model and a trained BackNet subnet model, to the one or more client computing devices.

Verifiable deep learning training service

Mechanisms are provided for detecting abnormal system call sequences in a monitored computing environment. The mechanisms receive, from a computing system resource of the monitored computing environment, a system call of an observed system call sequence for evaluation. A trained recurrent neural network (RNN), trained to predict system call sequences, processes the system call to generate a prediction of a subsequent system call in a predicted system call sequence. Abnormal call sequence logic compares the subsequent system call in the predicted system call sequence to an observed system call in the observed system call sequence and identifies a difference between the predicted system call sequence and the observed system call sequence based on results of the comparing. The abnormal call sequence logic generates an alert notification in response to identifying the difference.

Recurrent neural network based anomaly detection

Malware is detected and mitigated by differentiating HTTP error generation patterns between errors generated by malware, and errors generated by benign users/software. In one embodiment, a malware detector system receives traffic that includes HTTP errors and successful HTTP requests. Error traffic and the successful request traffic are segmented for further analysis. The error traffic is supplied to a clustering component, which groups the errors, e.g., based on their URI pages and parameters. During clustering, various statistical features are extracted (as feature vectors) from one or more perspectives, namely, error provenance, error generation, and error recovery. The feature vectors are supplied to a classifier component, which is trained to distinguish malware-generated errors from benign errors. Once trained, the classifier takes an error cluster and its surrounding successful HTTP requests as inputs, and it produces a verdict on whether a particular cluster is malicious. The classifier output then drives an automated mitigation operation.

Detecting and mitigating malware by evaluating HTTP errors

Mechanisms are provided to implement an enhanced privacy deep learning system framework (hereafter 'framework'). The framework receives, from a client computing device, an encrypted first subnet model of a neural network, where the first subnet model is one partition of multiple partitions of the neural network. The framework loads the encrypted first subnet model into a trusted execution environment (TEE) of the framework, decrypts the first subnet model, within the TEE, and executes the first subnet model within the TEE. The framework receives encrypted input data from the client computing device, loads the encrypted input data into the TEE, decrypts the input data, and processes the input data in the TEE using the first subnet model executing within the TEE.

Privacy enhancing deep learning cloud service using a trusted execution environment

One embodiment provides a method comprising receiving a training set comprising a plurality of data points, where a neural network is trained as a classifier based on the training set. The method further comprises, for each data point of the training set, classifying the data point with one of a plurality of classification labels using the trained neural network, and recording neuronal activations of a portion of the trained neural network in response to the data point. The method further comprises, for each classification label that a portion of the training set has been classified with, clustering a portion of all recorded neuronal activations that are in response to the portion of the training set, and detecting one or more poisonous data points in the portion of the training set based on the clustering.

Detecting poisoning attacks on neural networks by activation clustering

An automated method for cyberattack detection and prevention in an endpoint. The technique monitors and protects the endpoint by recording inter-process events, creating an inter-process activity graph based on the recorded inter-process events, matching the inter-process activity (as represented in the activity graph) against known malicious or suspicious behavior (as embodied in a set of one or more pattern graphs), and performing a post-detection operation in response to a match between an inter-process activity and a known malicious or suspicious behavior pattern. Preferably, matching involves matching a subgraph in the activity graph with a known malicious or suspicious behavior pattern as represented in the pattern graph. During this processing, preferably both direct and indirect inter-process activities at the endpoint (or across a set of endpoints) are compared to the known behavior patterns. The approach herein provides for systematic modeling of inter-process behaviors for characterizing malicious or suspicious patterns among processes.

Endpoint inter-process activity extraction and pattern matching

A framework to accurately and quickly verify the ownership of remotely-deployed deep learning models is provided without affecting model accuracy for normal input data. The approach involves generating a watermark, embedding the watermark in a local deep neural network (DNN) model by learning, namely, by training the local DNN model to learn the watermark and a predefined label associated therewith, and later performing a black-box verification against a remote service that is suspected of executing the DNN model without permission. The predefined label is distinct from a true label for a data item in training data for the model that does not include the watermark. Black-box verification includes simply issuing a query that includes a data item with the watermark, and then determining whether the query returns the predefined label.

Protecting deep learning models using watermarking

A computer-implemented method, a computer program product, and a computer system. The computer system installs and configures a virtual imitating resource in the computer system, wherein the virtual imitating resource imitates a set of resources in the computer system. Installing and configuring the virtual imitating resource includes modifying respective values of an installed version of the virtual imitating resource for an environment of the computer system, determining whether the virtual imitating resource is a static imitating resource or a dynamic imitating resource, and comparing a call graph of the evasive malware with patterns of dynamic imitating resources on a database. The computer system returns a response from an appropriate element of the virtual imitating resource, in response to a call from the evasive malware to a real computing resource, return, by the computer system.

Deactivating evasive malware

Mechanisms are provided for evaluating a trained machine learning model to determine whether the machine learning model has a backdoor trigger. The mechanisms process a test dataset to generate output classifications for the test dataset, and generate, for the test dataset, gradient data indicating a degree of change of elements within the test dataset based on the output generated by processing the test dataset. The mechanisms analyze the gradient data to identify a pattern of elements within the test dataset indicative of a backdoor trigger. The mechanisms generate, in response to the analysis identifying the pattern of elements indicative of a backdoor trigger, an output indicating the existence of the backdoor trigger in the trained machine learning model.

Using gradients to detect backdoors in neural networks

Mechanisms are provided for protecting a neural network model against model inversion attacks. The mechanisms generate a decoy dataset comprising decoy data for each class recognized by a neural network model. The mechanisms further configure the neural network model to generate a modified output based on the decoy dataset that directs a gradient of the modified output to the decoy dataset. The neural network model receives and process input data to generate an actual output. The neural network model modifies one or more actual elements of the actual output to be one or more corresponding modified elements of the modified output, and returns the one or more corresponding modified elements, instead of the one or more actual elements, to the source computing device.

Growing community of inventors

White Plains, NY, United States of America

Jialong Zhang