The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Apr. 15, 2014
Filed:
Dec. 16, 2009
Donghui Feng, Glendale, CA (US);
Sveva Besana, Los Angeles, CA (US);
Remi Zajac, Pasadena, CA (US);
Yellowpages.com LLC, Glendale, CA (US);
Abstract
Systems and computer-implemented methods include a validation phase in which validation projects are submitted to an online crowdsourcing system. The validation projects each have different combinations of the two validation control parameters, payment per human intelligence task ('HIT') and number of solutions per HIT. By analyzing the solutions for the validation projects, final control parameters can be selected from the validation project having the best combination of high quality, low cost, and short time to completion. The final control parameters can be submitted along with a final project to the online crowdsourcing system. In addition, a novel outlier detection algorithm enables the system to detect outlier workers based on the distribution of non-experts' agreement probabilities, reject outlier workers' results, and thus save labeling cost.