Microsoft Technology Licensing, LLC

Akshay Krishnamurthy

Miroslav Dudik

Maria Dimakopoulou

Yi Su

Off-policy evaluation of a new 'target' policy is performed using historical data gathered based on a previous 'logging' policy to estimate the performance of the target policy. An estimator may be used, wherein either a quality-based estimator or a quality-agnostic estimator is used to weight the difference between an observed reward in the historical data and an estimated reward generated by the target policy. A quality-agnostic estimator may be used to evaluate an importance weight according to a threshold. In such examples, when the importance weight exceeds the threshold, the quality-agnostic estimator clips the importance weight at the threshold, thereby providing an fixed upper bound irrespective of the quality of the reward predictor. In other examples, a quality-based estimator is used, in which an upper bound incorporates the quality of the reward predictor in order to modify an importance weight used by the estimator.

Growing community of inventors

Brooklyn, NY, United States of America

Akshay Krishnamurthy