International Business Machines Corporation

Taesung Lee

Ian M Molloy

Youngja Park

Benjamin J Edwards

Suresh N Chari

Jialong Zhang

Heqing Huang

Dong Su

Jiyong Jang

Bryant R Chen

Kevin Eykholt

Heiko Hary Ludwig

Josyula R Rao

Zhongshu Gu

Wilka Carvalho

Petr Novotny

Qi Zhang

Douglas Lee Schales

Lei Yu

Wenjie Wang

Farhan Tejani

Kathrin Grosse

Ebube Chuba

Shiqi Wang

Arpita Roy

A processor-implemented method generates adversarial example objects. One or more processors represent an adversarial input generation process as a graph. The processor(s) explore the graph, such that a sequence of edges on the graph are explored. The processor(s) create, based on the exploring, an adversarial example object, and utilize the created adversarial example object to harden an existing process model against vulnerabilities.

Graph exploration framework for adversarial example generation

Adaptive verifiable training enables the creation of machine learning models robust with respect to multiple robustness criteria. In general, such training exploits inherent inter-class similarities within input data and enforces multiple robustness criteria based on this information. In particular, the approach exploits pairwise class similarity and improves the performance of a robust model by relaxing robustness constraints for similar classes and increasing robustness constraints for dissimilar classes. Between similar classes, looser robustness criteria (i.e., smaller ∈) are enforced so as to minimize possible overlap when estimating the robustness region during verification. Between dissimilar classes, stricter robustness regions (i.e., larger ∈) are enforced. If pairwise class relationships are not available initially, preferably they are generated by receiving a pre-trained classifier and then applying a clustering algorithm (e.g., agglomerative clustering) to generate them. Once pre-defined or computed pairwise relationships are available, several grouping methods are provided to create classifiers for multiple robustness criteria.

Adaptive robustness certification against adversarial examples

Mechanisms for performing intelligent federated machine learning (ML) model updates are provided. A plurality of ML model updates, and a plurality of dataset sketch commitment data structures (sketches), are received from a plurality of participant computing systems. Each sketch provides statistical characteristics of a corresponding local dataset used by a corresponding participant to train a local ML model. A potentially malicious participant identification operation is performed based on an analysis of the plurality of sketches to identify one or more potentially malicious participants based on differences in sketches. ML model updates received from participant computing systems identified as potentially malicious participants are discarded to thereby generate a modified set of updates. The federated ML computer model is updated based on the modified set of updates.

Federated learning with dataset sketch commitment based malicious participant identification

Mechanisms are provided for obfuscating a trained configuration of a trained machine learning model. A trained machine learning model processes input data to generate an initial output vector having classification values for each of the plurality of predefined classes. A perturbation insertion engine determines a subset of classification values in the initial output vector into which to insert perturbations. A perturbation insertion engine modifies classification values in the subset of classification values by inserting a perturbation in a function associated with generating the output vector for the classification values in the subset of classification values, to thereby generate a modified output vector. The trained machine learning model outputs the modified output vector. The perturbation modifies the subset of classification values to obfuscate the trained configuration of the trained machine learning model while maintaining accuracy of classification of the input data.

Dynamic gradient deception against adversarial examples in machine learning models

One or more computer processors determine a tolerance value, and a norm value associated with an untrusted model and an adversarial training method. The one or more computer processors generate a plurality of interpolated adversarial images ranging between a pair of images utilizing the adversarial training method, wherein each image in the pair of images is from a different class. The one or more computer processors detect a backdoor associated with the untrusted model utilizing the generated plurality of interpolated adversarial images. The one or more computer processors harden the untrusted model by training the untrusted model with the generated plurality of interpolated adversarial images.

Adversarial interpolation backdoor detection

A behavioral biometrics deep learning (BBDL) pipeline is provided, comprising a plurality of stages of machine learning computer models that operate to provide a behavioral biometric based authenticator operating based on spatiotemporal input data. The BBDL pipeline receives spatiotemporal input data over a plurality of time intervals, each time interval having a corresponding subset of the spatiotemporal input data. For each time interval, machine learning computer model(s) of a corresponding stage process a subset of the spatiotemporal input data corresponding to the time interval to generate an output vector having values indicative of an internal representation of spatiotemporal traits of the entity. Output vectors are accumulated across the plurality of stages of the BBDL pipeline to generate a final output vector indicative of the spatiotemporal traits of the entity represented in the spatiotemporal input data. The entity is authenticated based on the final output vector.

Spatiotemporal deep learning for behavioral biometrics

A method, apparatus and computer program product to defend learning models that are vulnerable to adversarial example attack. It is assumed that data (a 'dataset') is available in multiple modalities (e.g., text and images, audio and images in video, etc.). The defense approach herein is premised on the recognition that the correlations between the different modalities for the same entity can be exploited to defend against such attacks, as it is not realistic for an adversary to attack multiple modalities. To this end, according to this technique, adversarial samples are identified and rejected if the features from one (the attacked) modality are determined to be sufficiently far away from those of another un-attacked modality for the same entity. In other words, the approach herein leverages the consistency between multiple modalities in the data to defend against adversarial attacks on one modality.

Using multimodal model consistency to detect adversarial attacks

Mechanisms are provided for obfuscating training of trained cognitive model logic. The mechanisms receive input data for classification into one or more classes in a plurality of predefined classes as part of a cognitive operation of the cognitive system. The input data is processed by applying a trained cognitive model to the input data to generate an output vector having values for each of the plurality of predefined classes. A perturbation insertion engine modifies the output vector by inserting a perturbation in a function associated with generating the output vector, to thereby generate a modified output vector. The modified output vector is then output. The perturbation modifies the one or more values to obfuscate the trained configuration of the trained cognitive model logic while maintaining accuracy of classification of the input data.

Protecting cognitive systems from model stealing attacks

A neural network is augmented to enhance robustness against adversarial attack. In this approach, a fully-connected additional layer is associated with a last layer of the neural network. The additional layer has a lower dimensionality than at least one or more intermediate layers. After sizing the additional layer appropriately, a vector bit encoding is applied. The encoding comprises an encoding vector for each output class. Preferably, the encoding is an n-hot encoding, wherein n represents a hyperparameter. The resulting neural network is then trained to encourage the network to associated features with each of the hot positions. In this manner, the network learns a reduced feature set representing those features that contain a high amount of information with respect to each output class, and/or to learn constraints between those features and the output classes. The trained neural network is used to perform a classification that is robust against adversarial examples.

Constraining neural networks for robustness through alternative encoding

Mechanisms are provided to implement a hardened ensemble artificial intelligence (AI) model generator. The hardened ensemble AI model generator co-trains at least two AI models. The hardened ensemble AI model generator modifies, based on a comparison of the at least two AI models, a loss surface of one or more of the at least two AI models to prevent an adversarial attack on one AI model, in the at least two AI models, transferring to another AI model in the at least two AI models, to thereby generate one or more modified AI models. At least one of the one or more modified AI models then processes an input to generate an output result.

Training diverse and robust ensembles of artificial intelligence computer models

Mechanisms are provided for detecting abnormal system call sequences in a monitored computing environment. The mechanisms receive, from a computing system resource of the monitored computing environment, a system call of an observed system call sequence for evaluation. A trained recurrent neural network (RNN), trained to predict system call sequences, processes the system call to generate a prediction of a subsequent system call in a predicted system call sequence. Abnormal call sequence logic compares the subsequent system call in the predicted system call sequence to an observed system call in the observed system call sequence and identifies a difference between the predicted system call sequence and the observed system call sequence based on results of the comparing. The abnormal call sequence logic generates an alert notification in response to identifying the difference.

Recurrent neural network based anomaly detection

Mechanisms are provided to determine a susceptibility of a trained machine learning model to a cybersecurity threat. The mechanisms execute a trained machine learning model on a test dataset to generate test results output data, and determine an overfit measure of the trained machine learning model based on the generated test results output data. The overfit measure quantifies an amount of overfitting of the trained machine learning model to a specific sub-portion of the test dataset. The mechanisms apply analytics to the overfit measure to determine a susceptibility probability that indicates a likelihood that the trained machine learning model is susceptible to a cybersecurity threat based on the determined amount of overfitting of the trained machine learning model. The mechanisms perform a corrective action based on the determined susceptibility probability.

Measuring overfitting of machine learning computer model and susceptibility to security threats

Mechanisms are provided to implement a hardened neural network framework. A data processing system is configured to implement a hardened neural network engine that operates on a neural network to harden the neural network against evasion attacks and generates a hardened neural network. The hardened neural network engine generates a reference training data set based on an original training data set. The neural network processes the original training data set and the reference training data set to generate first and second output data sets. The hardened neural network engine calculates a modified loss function of the neural network, where the modified loss function is a combination of an original loss function associated with the neural network and a function of the first and second output data sets. The hardened neural network engine trains the neural network based on the modified loss function to generate the hardened neural network.

Growing community of inventors

Ridgefield, CT, United States of America

Taesung Lee