Google Inc.

Badih Ghazi

Pasin Manurangsi

Shanmugasundaram Ravikumar

Rasmus Pagh

Ameya Avinash Velingker

Noah Zeger Golowich

Nachiappan Valliappan

Junfeng He

Chiyuan Zhang

Amer Sinha

Pritish Kamath

Kewen Wu

Jelani Osei Nelson

Alisa Chang

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training neural networks with label differential privacy. One of the methods includes, for each training example: processing the network input in the training example using the neural network in accordance with the values of the network parameters as of the beginning of the training iteration to generate a network output, generating a private network output for the training example from the target output in the training example and the network output for the training example, and generating a modified training example that includes the network input in the training example and the private network output for the training example; and training the neural network on at least the modified training examples to update the values of the network parameters.

Training neural networks with label differential privacy

Improved methods are provided for generating heatmaps or other summary map data from multiple users' data (e.g., probability distributions) in a manner that preserves the privacy of the users' data while also generating heatmaps that are visually similar to the 'true' heatmap. These methods include decomposing the average of the users' data (the ‘true’ heatmap) into multiple different spatial scales, injecting random noise into the data at the multiple different spatial scales, and then reconstructing the privacy-preserving heatmap based on the noisy multi-scale representations. The magnitude of the noise injected at each spatial scale is selected to ensure preservation of privacy while also resulting in heatmaps that are visually similar to the ‘true’ heatmap.

Differentially private heatmaps

An encoding process performed by a computing device (e.g., a user's private device) can include obtaining private data that includes a private value. According to an aspect of the present disclosure, the computing device can produce a plurality of messages that respectively comprise a plurality of message values, where a total sum of the plurality of message values approximates the private value, and where at least one of the plurality of message values is randomly selected. The device can provide the plurality of messages for aggregation with a plurality of additional messages respectively generated for a plurality of additional private values. For example, the messages can be transmitted to a shuffler model configured to shuffle the plurality of messages with the plurality of additional messages.

Scalable and differentially private distributed aggregation

An encoding method for enabling privacy-preserving aggregation of private data can include obtaining private data including a private value, determining a probabilistic status defining one of a first condition and a second condition, producing a multiset including a plurality of multiset values, and providing the multiset for aggregation with a plurality of additional multisets respectively generated for a plurality of additional private values. In response to the probabilistic status having the first condition, the plurality of multiset values is based at least in part on the private value, and in response to the probabilistic status having the second condition, the plurality of multiset values is a noise message. The noise message is produced based at least in part on a noise distribution that comprises a discretization of a continuous unimodal distribution supported on a range from zero to a number of multiset values included in the plurality of multiset values.

Pure differentially private algorithms for summation in the shuffled model

The present disclosure provides practical communication-efficient and low-error algorithms for aggregation of private data. For example, the proposed algorithms can be implemented in the shuffled DP model. Specific example operations that can be performed using the proposed algorithms include summation (e.g., binary summation, integer summation) and histograms over a moderate number of buckets. The proposed algorithms achieve accuracy that is arbitrarily close to that of central DP algorithms with an expected communication per user essentially matching what is needed without any privacy constraints.

Private counting from anonymous messages: near-optimal accuracy with vanishing communication overhead

Example techniques are provided for the task of differentially private clustering. For several basic clustering problems, including Euclidean DensestBall, 1-Cluster, k-means, and k-median, the present disclosure provides efficient differentially private algorithms that achieve essentially the same approximation ratios as those that can be obtained by any non-private algorithm, while incurring only small additive errors. This improves upon existing efficient algorithms that only achieve some large constant approximation factors.

Growing community of inventors

San Jose, CA, United States of America

Badih Ghazi