Microsoft Technology Licensing, LLC

Amazon Technologies, Inc.

Arnab Sinha

Chiyuan Huang

Juan Carlos Hernandez Munuera

Miguel Angel Hernandez Orozco

Varun Narayan Hegde

Lei Shi

Ravi Khandelwal

Rizwana Rizia

Methods, systems, and computer-readable media for offline evaluation of machine learning models with noise reduction are disclosed. A trigger computation system generates a plurality of experimental requests. The experimental requests do not represent unmodified requests received from clients in a production environment. At least one parameter value varies for individual ones of the experimental requests. The trigger computation system provides the experimental requests to a first machine learning model and a second machine learning model. The first machine learning model and the second machine learning model produce a set of results based at least in part on the experimental requests. The trigger computation system determines a reduced set of results for which the first machine learning model and the second machine learning model differ. An evaluation of the first machine learning model or the second machine learning model is performed using the reduced set of results.

Offline evaluation of machine learning models with noise reduction

Reductions in churn for assigning identifiers to entities in a knowledge graph enables several improvements to the functionality of the computing devices maintaining or accessing knowledge graphs. As the aliases or other terms used to identify a given entity change in response to updates to the knowledge graph, the identifiers assigned to various entities may change. For example, two individual entities conflated as one node may be split into two or two nodes merged into one in response to an update, and existing identifiers should be used to reduce churn. To select the existing identifiers to assign to a given updated entity, the aliases are clustered with the updated entities and the unique modal prior identifier is assigned as the identifier for the updated entity. Higher orders of modality are used to ensure as many existing identifiers are used before creating new identifiers.

Reducing churn in knowledge graphs

Growing community of inventors

Issaquah, WA, United States of America

Arnab Sinha