Oracle International Corporation

Oracle America, Inc.

Microsoft Technology Licensing, LLC

Amazon Technologies, Inc.

Sun Microsystems, Inc.

University of Rochester

Virendra J Marathe

David Dice

Alex Kogan

Nir N Shavit

Margo I Seltzer

Timothy Lawrence Harris

Victor M Luchangco

Steve Byan

Mark S Moir

Pallika Haridas Kanani

Daniel Peterson

Matej Pavlovic

Yosef Lev

Martin C Maas

Mihail-Igor Zablotchi

Achin Mishra

Amee Trivedi

Ahmed Alquraan

James R Larus

Sandhya Dwarkadas

Swetasudha Panda

Irina Calciu

Michael L Scott

Arrvindh Shriraman

Aleksandar Dragojević

Michael F Spear

Timorthy L Harris

Yihe Huang

Joseph H Izraelevitz

Peter Macko

Tito L Autrey, Iii

Anshuman Suri

Hamid Mozaffari

Hierarchical gradient averaging is performed as part of training a machine learning model to enforce subject level privacy. A sample of data items from a training data set is identified and respective gradients for the data items are determined. The gradients are then clipped. Each subject's clipped gradients in the sample are averaged. A noise value is added to a sum of the averaged gradients of each of the subjects in the sample. An average gradient for the entire sample is determined from the averaged gradients of the individual subjects with the added noise value. This average gradient for the entire sample is used for determining machine learning model updates.

Hierarchical gradient averaging for enforcing subject level privacy

User-level privacy preservation is implemented within federated machine learning. An aggregation server may distribute a machine learning model to multiple users each including respective private datasets. Individual users may train the model using the local, private dataset to generate one or more parameter updates. Prior to sending the generated parameter updates to the aggregation server for incorporation into the machine learning model, a user may modify the parameter updates by applying respective noise values to individual ones of the parameter updates to ensure differential privacy for the dataset private to the user. The aggregation server may then receive the respective modified parameter updates from the multiple users and aggregate the updates into a single set of parameter updates to update the machine learning model. The federated machine learning may further include iteratively performing said sending, training, modifying, receiving, aggregating and updating steps.

User-level privacy preservation for federated machine learning

Parameter permutation is performed for federated learning to train a machine learning model. Parameter permutation is performed by client systems of a federated machine learning system on updated parameters of a machine learning model that have been updated as part of training using local training data. An intra-model shuffling technique is performed at the client systems according to a shuffling pattern. Then, the encoded parameters are provided to an aggregation server using Private Information Retrieval (PIR) queries generated according to the shuffling pattern.

Federated learning by parameter permutation

Subject level privacy attack analysis for federated learning may be performed. A request that selects an analysis of one or more inference attacks may be received to determine a presence of data of a subject in a training set of a federated machine learning model. The selected inference attacks may be performed to determine the presence of the data of subject in the training set of the federated machine learning model. Respective success measurements may be generated for the selected inference attacks based on the performance of the selected inference attacks, which may then be provided.

Subject level privacy attack analysis for federated learning

NUMA-aware reader-writer locks may leverage lock cohorting techniques that introduce a synthetic level into the lock hierarchy (e.g., one whose nodes do not correspond to the system topology). The synthetic level may include a global reader lock and a global writer lock. A writer thread may acquire a node-level writer lock, then the global writer lock, and then the top-level lock, after which it may access a critical section protected by the lock. The writer may release the lock (if an upper bound on consecutive writers has been met), or may pass the lock to another writer (on the same node or a different node, according to a fairness policy). A reader may acquire the global reader lock (whether or not node-level reader locks are present), and then the top-level lock. However, readers may only hold these locks long enough to increment reader counts associated with them.

System and method for promoting reader groups for lock cohorting

Systems and methods are disclosed to improve disaster recovery by implementing a scalable low-loss disaster recovery for a data store. The disaster recovery system enables disaster recovery for a linearizable (e.g., externally consistent) distributed data store. The disaster recovery system also provides for a small lag on the backup site relative to the primary site, thereby reducing the data loss by providing a smaller data loss window compared to traditional disaster recovery techniques. The disaster recovery system implements a timestamp for log records based on a globally synchronized clock. The disaster recovery system also implements a watermark service that updates a global watermark timestamp that a backup node uses to apply log records.

Scalable low-loss disaster recovery for data stores

A computer system including one or more processors and persistent, word-addressable memory implements a persistent atomic multi-word compare-and-swap operation. On entry, a list of persistent memory locations of words to be updated, respective expected current values contained the persistent memory locations and respective new values to write to the persistent memory locations are provided. The operation atomically performs the process of comparing the existing contents of the persistent memory locations to the respective current values and, should they match, updating the persistent memory locations with the new values and returning a successful status. Should any of the contents of the persistent memory locations not match a respective current value, the operation returns a failed status. The operation is performed such that the system can recover from any failure or interruption by restoring the list of persistent memory locations.

Persistent multi-word compare-and-swap

Concurrent threads may be synchronized at the level of the memory words they access rather than at the level of the lock that protects the execution of critical sections. Each lock may be associated with an array of flags and each flag may indicate ownership of certain memory words. A pessimistic thread may set flags corresponding to memory words it is accessing in the critical section, while an optimistic thread may read the corresponding flag before any memory access to ensure that the flag is not set and that therefore the associated memory word is not being accessed by the other thread. Thus, optimistic threads that do not have conflicts with the pessimistic thread may not have to wait for the pessimistic thread to release the lock before proceeding.

Fine-grained hardware transactional lock elision

Herein are techniques for domain adaptation of a machine learning (ML) model. These techniques impose differential privacy onto federated learning by the ML model. In an embodiment, each of many client devices receive, from a server, coefficients of a general ML model. For respective new data point(s), each client device operates as follows. Based on the new data point(s), a respective private ML model is trained. Based on the new data point(s), respective gradients are calculated for the coefficients of the general ML model. Random noise is added to the gradients to generate respective noisy gradients. A combined inference may be generated based on: the private ML model, the general ML model, and one of the new data point(s). The noisy gradients are sent to the server. The server adjusts the general ML model based on the noisy gradients from the client devices. This client/server process may be repeated indefinitely.

Privacy preserving collaborative learning with domain adaptation

Undo logging for persistent memory transactions may permit concurrent transactions to write to the same persistent object. After an undo log record has been written, a single persist barrier may be issued. The tail pointer of the undo log may be updated after the persist barrier, and without another persist barrier, so the tail update may be persisted when the next log record is written and persisted. Undo logging for persistent memory transactions may rely on inferring the tail of an undo log after a failure rather than relying on a guaranteed correct tail pointer based on persisting the tail after every append. Additionally, transaction version numbers and checksum information may be stored to the undo log enabling failure recovery.

Persistent memory transactions with undo logging

A computer comprising one or more processors and memory may implement an atomic compare and swap (CAS) operation on multiple data elements. Each data element has a corresponding descriptor which includes a new value and a reference to a controlling descriptor for the CAS operation. The controlling descriptor includes a status value which indicates whether the CAS operation is in progress or has completed. The operation first allocates memory locations of the data elements by writing addresses of respective descriptors to the memory locations using CAS instructions. The operation then writes successful status to the status value of the controlling descriptor to indicate that the respective memory locations are no longer allocated. The operation then returns an indicator of successful completion without atomically updating the memory locations with the new values. Extensions are further described to implement CAS operations in non-volatile random access memories.

Efficient lock-free multi-word compare-and-swap

Growing community of inventors

Florence, MA, United States of America

Virendra J Marathe