Getting More for Less: Optimized Crowdsourcing with Dynamic Tasks and Goals

作者: Ari Kobren , Chun How Tan , Panagiotis Ipeirotis , Evgeniy Gabrilovich

DOI: 10.1145/2736277.2741681

关键词:

摘要: In crowdsourcing systems, the interests of contributing participants and system stakeholders are often not fully aligned. Participants seek to learn, be entertained, perform easy tasks, which offer them instant gratification; want users complete more difficult bring higher value crowdsourced application. We directly address this problem by presenting techniques that optimize process jointly maximizing user longevity in true derives from participation. first present models predict "survival probability" a at any given moment, is, probability will proceed next task offered system. then leverage survival model dynamically decide what assign motivating goals user. This allows us for short term (getting tasks done) long (keeping engaged longer periods time). show assigning significantly increases an extensive empirical evaluation, we observed our allocation strategy amount information collected up 117.8%. also explore utility with goals. demonstrate setting specific, static can highly detrimental long-term participation, as completion goal (e.g., earning badge) is common drop-off point many users. dynamically, conjunction judicious 249%, compared existing baselines use fixed objectives.

参考文章(28)
A. P. Dawid, A. M. Skene, Maximum Likelihood Estimation of Observer Error‐Rates Using the EM Algorithm Journal of The Royal Statistical Society Series C-applied Statistics. ,vol. 28, pp. 20- 28 ,(1979) , 10.2307/2346806
Pat Langley, Machine Learning for Adaptive User Interfaces Lecture Notes in Computer Science. pp. 53- 62 ,(1997) , 10.1007/3540634932_3
Praveen Paritosh, Peng Dai, Ed H. Chi, Jeffrey M. Rzeszotarski, Inserting Micro-Breaks into Crowdsourcing Workflows national conference on artificial intelligence. ,(2013)
Eric Horvitz, Andrew Mao, Ece Kamar, Why Stop Now? Predicting Worker Engagement in Online Crowdsourcing national conference on artificial intelligence. ,(2013)
Michel Krieger, Emily Margarete Stark, Scott R. Klemmer, Coordinating tasks on the commons: designing for personal goals, expertise and serendipity human factors in computing systems. pp. 1485- 1494 ,(2009) , 10.1145/1518701.1518927
Janette Lehmann, Mounia Lalmas, Elad Yom-Tov, Georges Dupret, Models of user engagement international conference on user modeling adaptation and personalization. pp. 164- 175 ,(2012) , 10.1007/978-3-642-31454-4_14
Magdalini Eirinaki, Michalis Vazirgiannis, Web mining for web personalization ACM Transactions on Internet Technology. ,vol. 3, pp. 1- 27 ,(2003) , 10.1145/643477.643478
M. B. WILK, R. GNANADESIKAN, Probability plotting methods for the analysis of data Biometrika. ,vol. 55, pp. 1- 17 ,(1968) , 10.1093/BIOMET/55.1.1
Luis von Ahn, Benjamin Maurer, Colin McMillen, David Abraham, Manuel Blum, reCAPTCHA: Human-Based Character Recognition via Web Security Measures Science. ,vol. 321, pp. 1465- 1468 ,(2008) , 10.1126/SCIENCE.1160379
David Easley, Arpita Ghosh, Incentives, gamification, and game theory Proceedings of the fourteenth ACM conference on Electronic commerce. ,vol. 4, pp. 359- 376 ,(2013) , 10.1145/2482540.2482571