Improving the Quality of Crowdsourced Image Labeling via Label Similarity

摘要: Crowdsourcing is an effective method to obtain large databases of manually-labeled images, which especially important for image understanding with supervised machine learning algorithms. However, several kinds tasks regarding labeling, e.g., dog breed recognition, it hard achieve high-quality results. Therefore, further optimizing crowdsourcing workflow mainly involves task allocation and result inference. For allocation, we design a two-round framework, contains smart decision mechanism based on information entropy determine whether perform the second round allocation. Regarding inference, after quantifying similarity all labels, two graphical models are proposed describe labeling process corresponding inference algorithms designed improve quality labeling. Extensive experiments real-world in Crowdflower synthesis datasets were conducted. The experimental results demonstrate superiority these methods comparison state-of-the-art methods.