International Business Machines Corporation

Green Market Square Limited

Heiko Hary Ludwig

Nathalie Baracaldo Angel

Robert John Moore

Bryant R Chen

Rui Zhang

Pawan Raghunath Chowdhary

Ali Anwar

Nagapramod S Mandagere

Asit Dan

Yi Zhou

Taiga Nakamura

Mohamed Mohamed

Nathalie Baracaldo-Angel

Gabriel Alatorre

Runhua Xu

Yang Song

Jonathan Frederick Brunn

Biplav Srivastava

Ian M Molloy

Mu Qiao

Evelyn Duesterwald

Alexander M Keller

Taesung Lee

William W Owen

Guangjie Ren

Samir Tata

Benjamin J Edwards

Hendrik Wagner

Robert Engel

Thomas Steinke

Amir J Safavi

Shashank Rajamoni

Stacey Truex

Henner Gimpel

Dimitri Kanevsky

Ramani Ranjan Routray

Larisa Shwartz

Genady Grabarnik

Sandeep Gopisetty

Jim Alain Laredo

Ehud Aharoni

Eric Kevin Butler

Tamar Eilam

Shun Jiang

Steve Richard White

Sunhwan Lee

John Arthur Pershing, Jr

Hovey Raymond Strong

Omri Soceanu

Michael H Kalantar

David M Ungar

Richard Pervin King

Margaret Helen Szymanski

Alexander V Konstantinou

Kamal Bhattacharya

Jialong Zhang

Alecio Pedro Delazari Binotto

Gautam Kar

Hayim Shaul

Peifeng Yin

Shubhi Asthana

Ami H Dewar

Andrea Schmidt

Eyal Kushnir

Aditya Agrawal

Rachael Marie Huston Dickens

Fernando Luiz Koch

Aladin Djuhera

Swanand Ravindra Kadhe

Hogun Park

Bruno Wasserman

Wilka Carvalho

Ebube Chuba

Farhan Ahmed

Obinna B Anya

Aikaterini Stamou

Liliana Pasquale

A computer-implemented method (CIM), according to one embodiment, includes performing a detecting and monitoring adversarial jamming (MDJ) process for an edge environment that includes a first edge device and a central hub. In response to a determination that a jamming attack event is likely to occur, a pre-emptive edge user device protection process (PPP) is caused to be deployed. In response to a determination that a jamming attack event has occurred, a post-process jammed data (PJD) process is caused to be deployed at a recipient of a wireless transmission subject to the jamming attack event. The CIM further includes generating, based on results of deploying the PPP and/or the PJD process, a first set of new parameters for a model of a device that sent the wireless transmission, and causing the first set of new parameters to be distributed to the model of the device that sent the wireless transmission.

Dynamically applied processes for mitigating adversarial jamming attack events

An example system includes a processor to compute a tensor of indicators indicating a presence of partial sums in an encrypted vector of indicators. The processor can also securely reorder an encrypted array based on the computed tensor of indicators to generate a reordered encrypted array.

Secure reordering using tensor of indicators

A plurality of public encryption keys are distributed to a plurality of participants in a federated learning system, and a first plurality of responses is received from the plurality of participants, where each respective response of the first plurality of responses was generated based on training data local to a respective participant of the plurality of participants and is encrypted using a respective public encryption key of the plurality of public encryption keys. A first aggregation vector is generated based on the first plurality of responses, and a first private encryption key is retrieved using the first aggregation vector. An aggregated model is then generated based on the first private encryption key and the first plurality of responses.

Privacy-preserving federated learning

Embodiments relate to an intelligent computer platform to build a federated learning framework including creating a hierarchy of machine learning models (MLMs). The hierarchy of MLMs has a primary MLM in a primary layer. Training the primary MLM includes capturing contributing model updates across at least one communication channel. A secondary MLM is created and logically positioned in a secondary layer of the hierarchy. The secondary MLM is operatively coupled to the primary MLM across the at least one communication channel. The created secondary MLM is initialized, including cloning weights and framework of the primary MLM into the secondary MLM, and populated with secondary data. The populated data has model updates local to the created secondary MLM. The secondary MLM is logically stored local to the secondary layer, and limits access to the secondary MLM to the secondary layer.

Building a federated learning framework

Provided are a computer program product, system, and method for generation of tasks and retraining machine learning modules to generate tasks based on feedback for the generated tasks. A machine learning module processes an input text message sent in the communication channel to output task information including an intended action and a set of associated users. A task message is generated including the output task information of a task to perform. The task message is sent to a user interface panel in a user computer. Feedback is received from the user computer on the output task information in the task message. The machine learning module is retrained to output task information from the input text message based on the feedback to reinforce likelihood correct task information is outputted and reinforce lower likelihood incorrect task information is outputted.

Automatic generation of tasks and retraining machine learning modules to generate tasks based on feedback for the generated tasks

One or more computer processors determine a tolerance value, and a norm value associated with an untrusted model and an adversarial training method. The one or more computer processors generate a plurality of interpolated adversarial images ranging between a pair of images utilizing the adversarial training method, wherein each image in the pair of images is from a different class. The one or more computer processors detect a backdoor associated with the untrusted model utilizing the generated plurality of interpolated adversarial images. The one or more computer processors harden the untrusted model by training the untrusted model with the generated plurality of interpolated adversarial images.

Adversarial interpolation backdoor detection

Computer-implemented methods, program products, and systems for provenance-based defense against poison attacks are disclosed. In one approach, a method includes: receiving observations and corresponding provenance data from data sources; determining whether the observations are poisoned based on the corresponding provenance data; and removing the poisoned observation(s) from a final training dataset used to train a final prediction model. Another implementation involves provenance-based defense against poison attacks in a fully untrusted data environment. Untrusted data points are grouped according to provenance signature, and the groups are used to train learning algorithms and generate complete and filtered prediction models. The results of applying the prediction models to an evaluation dataset are compared, and poisoned data points identified where the performance of the filtered prediction model exceeds the performance of the complete prediction model. Poisoned data points are removed from the set to generate a final prediction model.

Detecting and mitigating poison attacks using data provenance

Techniques regarding privacy preservation in a federated learning environment are provided. For example, one or more embodiments described herein can comprise a system, which can comprise a memory that can store computer executable components. The system can also comprise a processor, operably coupled to the memory, and that can execute the computer executable components stored in the memory. The computer executable components can comprise a plurality of machine learning components that can execute a machine learning algorithm to generate a plurality of model parameters. The computer executable components can also comprise an aggregator component that can synthesize a machine learning model based on an aggregate of the plurality of model parameters. The aggregator component can communicate with the plurality of machine learning components via a data privacy scheme that comprises a privacy process and a homomorphic encryption process in a federated learning environment.

Private and federated learning

Embodiments relate to a system, program product, and method for automatically determining which activation data points in a neural model have been poisoned to erroneously indicate association with a particular label or labels. A neural network is trained using potentially poisoned training data. Each of the training data points is classified using the network to retain the activations of the last hidden layer, and segment those activations by the label of corresponding training data. Clustering is applied to the retained activations of each segment, and a cluster assessment is conducted for each cluster associated with each label to distinguish clusters with potentially poisoned activations from clusters populated with legitimate activations. The assessment includes executing a set of analyses and integrating the results of the analyses into a determination as to whether a training data set is poisonous based on determining if resultant activation clusters are poisoned.

Automatically determining poisonous attacks on neural networks

One embodiment provides a method for federated learning across a plurality of data parties, comprising assigning each data party with a corresponding namespace in an object store, assigning a shared namespace in the object store, and triggering a round of federated learning by issuing a customized learning request to at least one data party. Each customized learning request issued to a data party triggers the data party to locally train a model based on training data owned by the data party and model parameters stored in the shared namespace, and upload a local model resulting from the local training to a corresponding namespace in the object store the data party is assigned with. The method further comprises retrieving, from the object store, local models uploaded to the object store during the round of federated learning, and aggregating the local models to obtain a shared model.

Parameter sharing in federated learning

Systems, computer-implemented methods, and computer program products that can facilitate detection of an adversarial backdoor attack on a trained model at inference time are provided. According to an embodiment, a system can comprise a memory that stores computer executable components and a processor that executes the computer executable components stored in the memory. The computer executable components can comprise a log component that records predictions and corresponding activation values generated by a trained model based on inference requests. The computer executable components can further comprise an analysis component that employs a model at an inference time to detect a backdoor trigger request based on the predictions and the corresponding activation values. In some embodiments, the log component records the predictions and the corresponding activation values from one or more layers of the trained model.

Detection of an adversarial backdoor attack on a trained model at inference time

Systems and techniques that facilitate universal and efficient privacy-preserving vertical federated learning are provided. In various embodiments, a key distribution component can distribute respective feature-dimension public keys and respective sample-dimension public keys to respective participants in a vertical federated learning framework governed by a coordinator, wherein the respective participants can send to the coordinator respective local model updates encrypted by the respective feature-dimension public keys and respective local datasets encrypted by the respective sample-dimension public keys. In various embodiments, an inference prevention component can verify a participant-related weight vector generated by the coordinator, based on which the key distribution component can distribute to the coordinator a functional feature-dimension secret key that can aggregate the encrypted respective local model updates into a sample-related weight vector. In various embodiments, the inference prevention component can verify the sample-related weight vector, based on which the key distribution component can distribute to the coordinator a functional sample-dimension secret key that can aggregate the encrypted respective local datasets into an update value for a global model.

Efficient private vertical federated learning

Embodiments relate to a system, program product, and method for processing an untrusted data set to automatically determine which data points there are poisonous. A neural network is trained network using potentially poisoned training data. Each of the training data points is classified using the network to retain the activations of at least one hidden layer, and segment those activations by the label of corresponding training data. Clustering is applied to the retained activations of each segment, and a clustering assessment is conducted to remove an identified cluster from the data set, form a new training set, and train a second neural model with the new training set. The removed cluster and corresponding data are applied to the trained second neural model to analyze and classify data in the removed cluster as either legitimate or poisonous.

Detecting backdoor attacks using exclusionary reclassification

Embodiments relate to a system, program product, and method for automatically determining which activation data points in a neural model have been poisoned to erroneously indicate association with a particular label or labels. A neural network is trained network using potentially poisoned training data. Each of the training data points is classified using the network to retain the activations of the last hidden layer, and segment those activations by the label of corresponding training data. Clustering is applied to the retained activations of each segment, and a cluster assessment is conducted for each cluster associated with each label to distinguish clusters with potentially poisoned activations from clusters populated with legitimate activations. The assessment includes analyzing, for each cluster, a distance of a median of the activations therein to medians of the activations in the labels.

Automatically determining whether an activation cluster contains poisonous data

Growing community of inventors

San Francisco, CA, United States of America

Heiko Hary Ludwig