Figure Eight Technologies, Inc.

Crowdflower, Inc.

Alexander Sorokin

Christopher R Van Pelt

Ryan J Cox

Lukas A Biewald

David Oleson

Greg Laughlin

John Le

Vaughn Hester

Systems and methods of dynamic optimization for data quality control in crowd sourcing tasks to crowd labor are disclosed. In one aspect, embodiments of the present disclosure include a method, which may be implemented on a system, for dynamically monitoring results received from workers for a task distributed for evaluation via a job distribution platform, incrementally assigning additional workers to the task using the results and continuously monitoring additional results to assign any additional workers if needed to meet a quality metric for the task.

Dynamic optimization for data quality control in crowd sourcing tasks to crowd labor

Systems and methods for evaluating a worker in performing crowd sourced tasks and providing in-task training are disclosed. In one aspect, embodiments of the present disclosure include a method, which may be implemented on a system, for selecting a job distributed through a job distribution platform for workers to work on, for use to generate a test task, the job being associated with a known correct result, associating a manipulated result, known to be an incorrect result for the job, with the job to generate the test task, and/or presenting the job with the manipulated result as the test task to a worker for evaluation of work quality of the worker. The job distribution platform crowd sources tasks online to workers to work on via their respective computing devices.

Evaluating a worker in performing crowd sourced tasks and providing in-task training through programmatically generated test tasks

Growing community of inventors

Foster City, CA, United States of America

Alexander Sorokin