Using cognitive models to combine probability estimates

摘要: We demonstrate the usefulness of cognitive models for combining human estimates probabilities in two experiments. Thefirst experiment involves people’s general knowledge questions such as “What percentage world’s population speaks English a first language?” The second football (soccer) games, is probability team leading 1–0 at half time will win game?”, with ground truths based on analysis large corpus games played past decade. In both experiments, we collect estimates, and develop model estimation process, including assumptions about calibration individual differences. show that approach outperforms standard statistical aggregation methods like mean median experiments and, unlike most previous related work, able to make good predictions fully unsupervised setting. also parameters inferred part modeling, involving expertise, provide useful measures characteristics individuals. argue has advantage aggregating over latent rather than observed emphasize it can be applied predictive settings where answers are not yet available.