Microsoft Technology Licensing, LLC

David Benjamin Levitan

Girish Kumar

Julia S McANALLEN

Robert Alexander Sim

Christopher Lawrence Laterza

Seyedeh Hoda Shajari

Huseyin Atahan Inan

Xiang Yue

Jiantao Pan

Improved solutions for dataset clustering and evaluation are disclosed. Examples cluster a set of documents into set of clusters using a language model, in an iterative process. In second and later clustering tasks, the current cluster titles and descriptions are provided in the language model prompt, to avoid near-duplications. Upon determining that the set of clusters is sufficiently complete and representative of the set of documents, the tasking switches to classification of the set of documents into the set of clusters using a language model. Classification continues until a sufficient percentage of the set of documents is classified. Some examples use batching, to avoid overloading the language model(s). In some examples, different language models are used for clustering and classification. Some examples use intruder detection to determine the quality of the clustering. This process provides superior performance on classifying documents having little configuration control, such as website feedback from consumers.

Dataset clustering and evaluation

Systems and methods are directed to building annotated models based on eyes-off data. Specifically, a synthetic data generation model is trained and used to further train a target model. The synthetic data generation model is trained within an eyes-off environment using an anonymity technique on confidential data. The synthetic data generation model is then used to create synthetic data that closely represents the confidential data but without any specific details that can be linked back to the confidential data. The synthetic data is then annotated and used to train the target model within an eyes-on environment. Subsequently, the target model is deployed back within the eyes-off environment to classify the confidential data.

Building annotated models based on eyes-off data

A method and system for generating synthetic privacy preserving training data for training a language classifier machine-learning (ML) model includes receiving a request to generate the synthetic privacy-preserving training data for the language classifier ML model, retrieving labeled training data associated with training the language classifier ML model, providing the labeled training data, one or more privacy parameters, and a domain type associated with the labeled training data to a synthetic data generation ML model, the synthetic data generation ML model being configured to generate synthetic training data in a privacy-persevering manner, receiving synthetic privacy-preserving training data as an output from the synthetic data generation ML model, and providing the synthetic privacy preserving training data to the language classifier ML model for training the language classifier ML model in classifying text.

Generating private synthetic training data for training machine-learning models

Growing community of inventors

Bothell, WA, United States of America

David Benjamin Levitan