The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Oct. 24, 2023
Filed:
Oct. 18, 2019
Microsoft Technology Licensing, Llc, Redmond, WA (US);
Miroslav Dudik, Montclair, NJ (US);
Akshay Krishnamurthy, Brooklyn, NY (US);
Maria Dimakopoulou, Sunnyvale, CA (US);
Yi Su, Ithaca, NY (US);
Microsoft Technology Licensing, LLC, Redmond, WA (US);
Abstract
Off-policy evaluation of a new 'target' policy is performed using historical data gathered based on a previous 'logging' policy to estimate the performance of the target policy. An estimator may be used, wherein either a quality-based estimator or a quality-agnostic estimator is used to weight the difference between an observed reward in the historical data and an estimated reward generated by the target policy. A quality-agnostic estimator may be used to evaluate an importance weight according to a threshold. In such examples, when the importance weight exceeds the threshold, the quality-agnostic estimator clips the importance weight at the threshold, thereby providing an fixed upper bound irrespective of the quality of the reward predictor. In other examples, a quality-based estimator is used, in which an upper bound incorporates the quality of the reward predictor in order to modify an importance weight used by the estimator.