A statistical model for aggregating judgments by incorporating peer predictions

摘要: We propose a probabilistic model to aggregate the answers of respondents answering multiple-choice questions. The does not assume that everyone has access same information, and so consensus answer is correct. Instead, it infers most probable world state, even if only minority vote for it. Each respondent modeled as receiving signal contingent on actual using this both determine their own predict given by others. By incorporating respondent's predictions others' answers, latent parameters corresponding prior over states probability different signals being received in all possible states, including counterfactual ones. Unlike other models aggregation, our applies single multiple questions, which case estimates each expertise. shows good performance, compared number models, data from seven studies covering types