The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Apr. 19, 2022
Filed:
Dec. 30, 2019
Scale Ai, Inc., San Francisco, CA (US);
Nathaniel John Herman, San Francisco, CA (US);
Akshat Bubna, San Francisco, CA (US);
Alexandr Wang, San Francisco, CA (US);
Shariq Shahab Hashme, San Francisco, CA (US);
Samuel J. Clearman, San Francisco, CA (US);
Liren Tu, Danville, CA (US);
Jeffrey Zhihong Li, San Francisco, CA (US);
James Lennon, San Francisco, CA (US);
SCALE AI, INC., San Francisco, CA (US);
Abstract
One embodiment of the present invention sets forth a technique for evaluating labeled data. The technique includes selecting, from a set of labels for a data sample, a subset of the labels representing non-outliers in a distribution of values in the set of labels. The technique also includes aggregating the subset of the labels into a benchmark for the data sample. The technique further includes generating, based on a comparison between the benchmark and an additional label, a benchmark score associated with the data sample, and generating a set of performance metrics for labeling the data sample based on the benchmark score.