The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Apr. 16, 2024
Filed:
Apr. 01, 2021
Microsoft Technology Licensing, Llc, Redmond, WA (US);
Aleksander Fabijan, Bellevue, WA (US);
Trevor Craig Blanarik, Seattle, WA (US);
Kewei Chen, Bellevue, WA (US);
Ruhan Zhang, Bellevue, WA (US);
Adam Marc Gustafson, Seattle, WA (US);
Venkata Kavitha Budumuri, Kirkland, WA (US);
Stephen James Hunt, Bellevue, WA (US);
Maxwell Campbell Caughron, Kenmore, WA (US);
Vaibhav Kumar Bajpai, Bothell, WA (US);
Microsoft Technology Licensing, LLC, Redmond, WA (US);
Abstract
A sample ratio mismatch (SRM) analyzer receives data from an online controlled experiment (OCE) and provides information to help determine a root cause of an SRM. The SRM analyzer may identify one or more segments in the data that include an SRM and may determine whether a triggered scorecard of the OCE includes an SRM. The data may include one or more scorecards. The SRM analyzer may determine whether each scorecard has an SRM. The SRM analyzer may test a difference in proportion of users assigned to treatment between a last scorecard without an SRM and a first scorecard with an SRM. If the difference in proportion is statistically meaningful, the SRM analyzer may determine that the SRM arose after the last scorecard. If the difference in proportions is not statistically meaningful, the SRM analyzer may determine that the SRM existed from a beginning of the OCE.