Streamlined creation and utilization of reference human intelligence tasks

摘要: Reference intelligence tasks are automatically generated for subsequent utilization in crowdsourced processing of tasks. categorized into predetermined categories, including categories defined by an intended such A trusted set workers provided with and, if a specified number those reach consensus as to what is appropriate answer, then answer definitive answer. Conversely, no initially reached, additional can be utilized determine if, combination, reached. To frame categorization considerations the workers, they statistics relevant categorizations. An automatic category assignment, or reassignment, performed necessary. Additionally, maximum amount time which generation reference complete is, optionally, established.