On the impossibility of convex inference in human computation

作者: Nihar Shah , Dengyong Zhou , None

DOI:

关键词:

摘要: Human computation or crowdsourcing involves joint inference of the ground-truth-answers and worker-abilities by optimizing an objective function, for instance, maximizing data likelihood based on assumed underlying model. A variety methods have been proposed in literature to address this problem. As far as we know, none functions existing is convex. In machine learning applied statistics, a convex function such support vector machines (SVMs) generally preferred, since it can leverage high-performance algorithms rigorous guarantees established extensive optimization. One may thus wonder if there exists meaningful problem human computation. paper, investigate convexity issue We take axiomatic approach formulating set axioms that impose two mild natural assumptions inference. Under these axioms, show unfortunately impossible ensure On other hand, interestingly, absence requirement model "spammers", one construct reasonable guarantee

参考文章(27)
A. P. Dawid, A. M. Skene, Maximum Likelihood Estimation of Observer Error‐Rates Using the EM Algorithm Journal of The Royal Statistical Society Series C-applied Statistics. ,vol. 28, pp. 20- 28 ,(1979) , 10.2307/2346806
Nilesh Dalvi, Anirban Dasgupta, Ravi Kumar, Vibhor Rastogi, Aggregating crowdsourced binary ratings Proceedings of the 22nd international conference on World Wide Web - WWW '13. pp. 285- 294 ,(2013) , 10.1145/2488388.2488414
Yarun Luon, Christina Aperjis, Bernardo A. Huberman, Rankr: A Mobile System for Crowdsourcing Opinions mobile computing, applications, and services. pp. 20- 31 ,(2011) , 10.1007/978-3-642-32320-1_2
Toshihiro Kamishima, Hisashi Kashima, Toshiko Matsui, Yukino Baba, Crowdsourcing Quality Control for Item Ordering Tasks national conference on artificial intelligence. ,(2013)
Gabriella Kazai, Jaap Kamps, Marijn Koolen, Natasa Milic-Frayling, Crowdsourcing for book search evaluation: impact of hit design on comparative system ranking international acm sigir conference on research and development in information retrieval. pp. 205- 214 ,(2011) , 10.1145/2009916.2009947
Arpita Ghosh, Satyen Kale, Preston McAfee, Who moderates the moderators?: crowdsourcing abuse detection in user-generated content electronic commerce. pp. 167- 176 ,(2011) , 10.1145/1993574.1993599
L. L. Thurstone, A law of comparative judgment Psychological Review. ,vol. 34, pp. 273- 286 ,(1994) , 10.1037/H0070288
Aditya Vempaty, Lav R. Varshney, Pramod K. Varshney, Reliable classification by unreliable crowds 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. pp. 5558- 5562 ,(2013) , 10.1109/ICASSP.2013.6638727
J. Bohannon, Social Science for Pennies Science. ,vol. 334, pp. 307- 307 ,(2011) , 10.1126/SCIENCE.334.6054.307