Apple Inc.

Abhishek Bhowmick

Umesh Suresh Vaishampayan

Andrew H Vyrros

Ryan M Rogers

Kevin W Decker

Gaurav Kapoor

Steven J Falkenburg

Conrad A Shultz

Mateusz Rajca

Steve Falkenburg

Julien F Freudiger

Lucas Olen Winstrom

Mona Chitnis

John Duchi

Matthew R Salesi

Stephen D Fleischer

Koray Mancuhan

Vignesh Jagadeesh

Kartik R Venkatraman

Gavin Barraclough

Sowmya Gopalan

Kalu Onuka Kalu

Guillaume Tartavel

Stephen Cosman

Alexis Hugo Louis Durocher

Christophe Dumez

Marcelo Lotif Araujo

Michael Chatzidakis

Thi Hai Van Do

Christophe P Dumez

Mateusz K Rajca

Embodiments described herein provide a technique to crowdsource labeling of training data for a machine learning model while maintaining the privacy of the data provided by crowdsourcing participants. Client devices can be used to generate proposed labels for a unit of data to be used in a training dataset. One or more privacy mechanisms are used to protect user data when transmitting the data to a server. The server can aggregate the proposed labels and use the most frequently proposed labels for an element as the label for the element when generating training data for the machine learning model. The machine learning model is then trained using the crowdsourced labels to improve the accuracy of the model.

Distributed labeling for supervised learning

The subject technology for maintaining differential privacy for database query results receives a query for a database that contains user data. The subject technology determines that the query is permitted for the database based at least in part on a privacy policy associated with the database. The subject technology determines that performing the query will not exceed a query budget for the database. The subject technology, when the query is permitted and performing the query will not exceed the query budget, performs the query on the database and receiving results from the query. The subject technology selects a differential privacy algorithm for the results based at least in part on a query type of the query. The subject technology applies the selected differential privacy algorithm to the results to generate differentially private results. The subject technology provides the differentially private results.

Maintaining differential privacy for database query results

Embodiments described herein provide for a non-transitory machine-readable medium storing instructions to cause one or more processors to receive, at a client device, a machine learning model from a server, detect a usage pattern for a content item, store an association between the content item and the detected usage pattern in local data, train the machine learning model using local data for the content item with the detected usage pattern to generate a trained machine learning model, generate an update for the machine learning model, privatize the update for the machine learning model, and transmit the privatized update for the machine learning model to the server.

User behavior model development with private federated learning

Embodiments described herein provide for a non-transitory machine-readable medium storing instructions to cause one or more processors to perform operations comprising receiving a machine learning model from a server at a client device, training the machine learning model using local data at the client device, generating an update for the machine learning model, the update including a weight vector that represents a difference between the received machine learning model and the trained machine learning model, privatizing the update for the machine learning model, and transmitting the privatized update for the machine learning model to the server.

Private federated learning with protection against reconstruction

Embodiments described herein ensure differential privacy when transmitting data to a server that estimates a frequency of such data amongst a set of client devices. The differential privacy mechanism may provide a predictable degree of variance for frequency estimations of data. The system may use a multibit histogram model or Hadamard multibit model for the differential privacy mechanism, both of which provide a predictable degree of accuracy of frequency estimations while still providing mathematically provable levels of privacy.

Differential privacy using a multibit histogram

Embodiments described herein provide a privacy mechanism to protect user data when transmitting the data to a server that estimates a frequency of such data amongst a set of client devices. In one embodiment, a differential privacy mechanism is implemented using a count-mean-sketch technique that can reduce resource requirements required to enable privacy while providing provable guarantees regarding privacy and utility. For instance, the mechanism can provide the ability to tailor utility (e.g. accuracy of estimations) against the resource requirements (e.g. transmission bandwidth and computation complexity).

User experience using privatized crowdsourced data

Embodiments described herein provide techniques to encode sequential data in a privacy preserving manner before the data is sent to a sequence learning server. The server can then determine aggregate trends within an overall set of users, without having any specific knowledge about the contributions of individual users. The server can be used to learn new words generated by user client devices in a crowdsourced manner while maintaining local differential privacy of client devices. The server can also learn other sequential data including typed, autocorrected, revised text sequences, sequences of application launches, sequences of purchases on an application store, or other sequences of activities that can be performed on an electronic device.

Privatized apriori algorithm for sequential data discovery

Embodiments described herein provide a privacy mechanism to protect user data when transmitting the data to a server that estimates a frequency of such data amongst a set of client devices. One embodiment uses a differential privacy mechanism to enhance a user experience by inferring potential user preferences from analyzing crowdsourced user interaction data. Based on a statistical analysis of user interactions in relation to various features or events, development efforts with respect to application behavior may be refined or enhanced. For example, user interactions in relation to the presentation of content such as content from online sources may be analyzed. Accordingly, presentation settings or preferences may be defined based on the crowdsourced user interaction data.

Embodiments described herein provide a privacy mechanism to protect user data when transmitting the data to a server that estimates a frequency of such data amongst a set of client devices. One embodiment uses a differential privacy mechanism to enhance a user experience by identifying particular websites that exhibit particular characteristics. In one embodiment, websites that are associated with a high resource consumption are identified. High resource consumption can be identified based on threshold of particular resources such as processor, memory, network bandwidth, and power usage.

Growing community of inventors

San Francisco, CA, United States of America

Abhishek Bhowmick