Scale Ai, Inc.

Liren Tu

Nathaniel John Herman

Akshat Bubna

Samuel Jacob Clearman

Shariq Shahab Hashme

Jeffrey Zhihong Li

Alexandr Wang

James Lennon

One embodiment of the present invention sets forth a technique for evaluating labeled data. The technique includes selecting, from a set of labels for a data sample, a subset of the labels representing non-outliers in a distribution of values in the set of labels. The technique also includes aggregating the subset of the labels into a benchmark for the data sample. The technique further includes generating, based on a comparison between the benchmark and an additional label, a benchmark score associated with the data sample, and generating a set of performance metrics for labeling the data sample based on the benchmark score.

Automatic benchmarking of labeling tasks

Growing community of inventors

Danville, CA, United States of America

Liren Tu