The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Sep. 30, 2025
Filed:
Feb. 09, 2023
Oracle International Corporation, Redwood Shores, CA (US);
Gioacchino Tangari, Sydney, AU;
Cong Duy Vu Hoang, Wantirna South, AU;
Mark Edward Johnson, Sydney, AU;
Poorya Zaremoodi, Melbourne, AU;
Nitika Mathur, Melbourne, AU;
Aashna Devang Kanuga, Foster City, CA (US);
Thanh Long Duong, Seabrook, AU;
ORACLE INTERNATIONAL CORPORATION, Redwood Shores, CA (US);
Abstract
Techniques are disclosed herein for calibrating confidence scores of a machine learning model trained to translate natural language to a meaning representation language. The techniques include obtaining one or more raw beam scores generated from one or more beam levels of a decoder of a machine learning model trained to translate natural language to a logical form, where each of the one or more raw beam scores is a conditional probability of a sub-tree determined by a heuristic search algorithm of the decoder at one of the one or more beam levels, classifying, by a calibration model, a logical form output by the machine learning model as correct or incorrect based on the one or more raw beam scores, and providing the logical form with a confidence score that is determined based on the classifying of the logical form.