International Business Machines Corporation

Arkose Labs Holdings, Inc.

Globalfoundries Inc.

Ian M Molloy

Suresh N Chari

Taesung Lee

Youngja Park

Dimitrios Pendarakis

Benjamin J Edwards

Ashish Kundu

Jialong Zhang

Josyula R Rao

Dong Su

Zhongshu Gu

Wilfried Teiken

Heqing Huang

Michael Vu Le

Ted A Habeck

Jiyong Jang

Lawrence H Koved

Bryant R Chen

Kevin Eykholt

Heiko Hary Ludwig

Pau-Chen Cheng

Gelareh Taban

Taemin Park

Wilka Carvalho

Jacob Tinkhauser

Marc Philippe Stoecklin

Hani Talal Jamjoom

Douglas Lee Schales

Christopher Gates

Bo Zhang

Hagai Aronowitz

Enriquillo Valdez

Sulakshan Vajipayajula

Paul Coccoli

Marc Stoecklin

James Brent Peterson

Priti P Patil

Wenjie Wang

Kathrin Grosse

Kushaal Veijay

Ebube Chuba

Shiqi Wang

Farhan Tejani

A processor-implemented method generates adversarial example objects. One or more processors represent an adversarial input generation process as a graph. The processor(s) explore the graph, such that a sequence of edges on the graph are explored. The processor(s) create, based on the exploring, an adversarial example object, and utilize the created adversarial example object to harden an existing process model against vulnerabilities.

Graph exploration framework for adversarial example generation

Adaptive verifiable training enables the creation of machine learning models robust with respect to multiple robustness criteria. In general, such training exploits inherent inter-class similarities within input data and enforces multiple robustness criteria based on this information. In particular, the approach exploits pairwise class similarity and improves the performance of a robust model by relaxing robustness constraints for similar classes and increasing robustness constraints for dissimilar classes. Between similar classes, looser robustness criteria (i.e., smaller ∈) are enforced so as to minimize possible overlap when estimating the robustness region during verification. Between dissimilar classes, stricter robustness regions (i.e., larger ∈) are enforced. If pairwise class relationships are not available initially, preferably they are generated by receiving a pre-trained classifier and then applying a clustering algorithm (e.g., agglomerative clustering) to generate them. Once pre-defined or computed pairwise relationships are available, several grouping methods are provided to create classifiers for multiple robustness criteria.

Adaptive robustness certification against adversarial examples

Mechanisms are provided for obfuscating a trained configuration of a trained machine learning model. A trained machine learning model processes input data to generate an initial output vector having classification values for each of the plurality of predefined classes. A perturbation insertion engine determines a subset of classification values in the initial output vector into which to insert perturbations. A perturbation insertion engine modifies classification values in the subset of classification values by inserting a perturbation in a function associated with generating the output vector for the classification values in the subset of classification values, to thereby generate a modified output vector. The trained machine learning model outputs the modified output vector. The perturbation modifies the subset of classification values to obfuscate the trained configuration of the trained machine learning model while maintaining accuracy of classification of the input data.

Dynamic gradient deception against adversarial examples in machine learning models

A method, apparatus and computer program product to protect a deep neural network (DNN) having a plurality of layers including one or more intermediate layers. In this approach, a training data set is received. During training of the DNN using the received training data set, a representation of activations associated with an intermediate layer is recorded. For at least one or more of the representations, a separate classifier (model) is trained. The classifiers, collectively, are used to train an outlier detection model. Following training, the outliner detection model is used to detect an adversarial input on the deep neural network. The outlier detection model generates a prediction, and an indicator whether a given input is the adversarial input. According to a further aspect, an action is taken to protect a deployed system associated with the DNN in response to detection of the adversary input.

Detecting adversary attacks on a deep neural network (DNN)

One or more computer processors determine a tolerance value, and a norm value associated with an untrusted model and an adversarial training method. The one or more computer processors generate a plurality of interpolated adversarial images ranging between a pair of images utilizing the adversarial training method, wherein each image in the pair of images is from a different class. The one or more computer processors detect a backdoor associated with the untrusted model utilizing the generated plurality of interpolated adversarial images. The one or more computer processors harden the untrusted model by training the untrusted model with the generated plurality of interpolated adversarial images.

Adversarial interpolation backdoor detection

A behavioral biometrics deep learning (BBDL) pipeline is provided, comprising a plurality of stages of machine learning computer models that operate to provide a behavioral biometric based authenticator operating based on spatiotemporal input data. The BBDL pipeline receives spatiotemporal input data over a plurality of time intervals, each time interval having a corresponding subset of the spatiotemporal input data. For each time interval, machine learning computer model(s) of a corresponding stage process a subset of the spatiotemporal input data corresponding to the time interval to generate an output vector having values indicative of an internal representation of spatiotemporal traits of the entity. Output vectors are accumulated across the plurality of stages of the BBDL pipeline to generate a final output vector indicative of the spatiotemporal traits of the entity represented in the spatiotemporal input data. The entity is authenticated based on the final output vector.

Spatiotemporal deep learning for behavioral biometrics

A method, apparatus and computer program product to defend learning models that are vulnerable to adversarial example attack. It is assumed that data (a 'dataset') is available in multiple modalities (e.g., text and images, audio and images in video, etc.). The defense approach herein is premised on the recognition that the correlations between the different modalities for the same entity can be exploited to defend against such attacks, as it is not realistic for an adversary to attack multiple modalities. To this end, according to this technique, adversarial samples are identified and rejected if the features from one (the attacked) modality are determined to be sufficiently far away from those of another un-attacked modality for the same entity. In other words, the approach herein leverages the consistency between multiple modalities in the data to defend against adversarial attacks on one modality.

Using multimodal model consistency to detect adversarial attacks

Using a deep learning inference system, respective similarities are measured for each of a set of intermediate representations to input information used as an input to the deep learning inference system. The deep learning inference system includes multiple layers, each layer producing one or more associated intermediate representations. Selection is made of a subset of the set of intermediate representations that are most similar to the input information. Using the selected subset of intermediate representations, a partitioning point is determined in the multiple layers used to partition the multiple layers into two partitions defined so that information leakage for the two partitions will meet a privacy parameter when a first of the two partitions is prevented from leaking information. The partitioning point is output for use in partitioning the multiple layers of the deep learning inference system into the two partitions.

System for measuring information leakage of deep learning models

Mechanisms are provided for obfuscating training of trained cognitive model logic. The mechanisms receive input data for classification into one or more classes in a plurality of predefined classes as part of a cognitive operation of the cognitive system. The input data is processed by applying a trained cognitive model to the input data to generate an output vector having values for each of the plurality of predefined classes. A perturbation insertion engine modifies the output vector by inserting a perturbation in a function associated with generating the output vector, to thereby generate a modified output vector. The modified output vector is then output. The perturbation modifies the one or more values to obfuscate the trained configuration of the trained cognitive model logic while maintaining accuracy of classification of the input data.

Protecting cognitive systems from model stealing attacks

A neural network is augmented to enhance robustness against adversarial attack. In this approach, a fully-connected additional layer is associated with a last layer of the neural network. The additional layer has a lower dimensionality than at least one or more intermediate layers. After sizing the additional layer appropriately, a vector bit encoding is applied. The encoding comprises an encoding vector for each output class. Preferably, the encoding is an n-hot encoding, wherein n represents a hyperparameter. The resulting neural network is then trained to encourage the network to associated features with each of the hot positions. In this manner, the network learns a reduced feature set representing those features that contain a high amount of information with respect to each output class, and/or to learn constraints between those features and the output classes. The trained neural network is used to perform a classification that is robust against adversarial examples.

Constraining neural networks for robustness through alternative encoding

Deep learning training service framework mechanisms are provided. The mechanisms receive encrypted training datasets for training a deep learning model, execute a FrontNet subnet model of the deep learning model in a trusted execution environment, and execute a BackNet subnet model of the deep learning model external to the trusted execution environment. The mechanisms decrypt, within the trusted execution environment, the encrypted training datasets and train the FrontNet subnet model and BackNet subnet model of the deep learning model based on the decrypted training datasets. The FrontNet subnet model is trained within the trusted execution environment and provides intermediate representations to the BackNet subnet model which is trained external to the trusted execution environment using the intermediate representations. The mechanisms release a trained deep learning model comprising a trained FrontNet subnet model and a trained BackNet subnet model, to the one or more client computing devices.

Verifiable deep learning training service

Mechanisms are provided to implement a hardened ensemble artificial intelligence (AI) model generator. The hardened ensemble AI model generator co-trains at least two AI models. The hardened ensemble AI model generator modifies, based on a comparison of the at least two AI models, a loss surface of one or more of the at least two AI models to prevent an adversarial attack on one AI model, in the at least two AI models, transferring to another AI model in the at least two AI models, to thereby generate one or more modified AI models. At least one of the one or more modified AI models then processes an input to generate an output result.

Training diverse and robust ensembles of artificial intelligence computer models

Mechanisms are provided for detecting abnormal system call sequences in a monitored computing environment. The mechanisms receive, from a computing system resource of the monitored computing environment, a system call of an observed system call sequence for evaluation. A trained recurrent neural network (RNN), trained to predict system call sequences, processes the system call to generate a prediction of a subsequent system call in a predicted system call sequence. Abnormal call sequence logic compares the subsequent system call in the predicted system call sequence to an observed system call in the observed system call sequence and identifies a difference between the predicted system call sequence and the observed system call sequence based on results of the comparing. The abnormal call sequence logic generates an alert notification in response to identifying the difference.

Recurrent neural network based anomaly detection

Anomalous control and data flow paths in a program are determined by machine learning the program's normal control flow paths and data flow paths. A subset of those paths also may be determined to involve sensitive data and/or computation. Learning involves collecting events as the program executes, and associating those event with metadata related to the flows. This information is used to train the system about normal paths versus anomalous paths, and sensitive paths versus non-sensitive paths. Training leads to development of a baseline 'provenance' graph, which is evaluated to determine 'sensitive' control or data flows in the &#x201c;normal&#x201d; operation. This process is enhanced by analyzing log data collected during runtime execution of the program against a policy to assign confidence values to the control and data flows. Using these confidence values, anomalous edges and/or paths with respect to the policy are identified to generate a &#x201c;program execution&#x201d; provenance graph associated with the policy.

Policy-based detection of anomalous control and data flow paths in an application program

Growing community of inventors

Chappaqua, NY, United States of America

Ian M Molloy