How do People Sort by Ratings

摘要: Sorting items by user rating is a fundamental interaction pattern of the modern Web, used to rank products (Amazon), posts (Reddit), businesses (Yelp), movies (YouTube), and more. To implement this pattern, designers must take in distribution ratings for each item define sensible total ordering over them. This challenging problem, since drawn from distinct sample population, rendering most straightforward method sorting --- comparing averages unreliable when samples are small or different sizes. Several statistical orderings binary have been proposed literature (e.g., based on Wilson score, Laplace smoothing), attempting account uncertainty introduced sampling. In paper, we study through lens human perception, ask "How do people sort ratings?" an online study, collected 48,000 item-ranking pairs 4,000 crowd workers along with 4,800 rationales, analyzed results understand how users make decisions rated items. Our shed light cognitive models employ choose between distributions, which sorts comparisons contentious, presentation information affects users' preferences.