Optimal crowd-powered rating and filtering algorithms

摘要: We focus on crowd-powered filtering, i.e., filtering a large set of items using humans. Filtering is one the most commonly used building blocks in crowdsourcing applications and systems. While solutions for exist, they make range implicit assumptions restrictions, ultimately rendering them not powerful enough real-world applications. describe two approaches to discard these restrictions: one, that carefully generalizes prior work, leading an optimal, but often-times intractable solution, another, provides novel way reasoning about strategies, sometimes suboptimal, efficiently computable solution (that asymptotically close optimal). demonstrate our techniques lead significant reductions error up 30% fixed cost over work application: peer evaluation online courses.