Learning from multiple annotators using kernel alignment

摘要: Abstract In a typical supervised learning scenario, it is supposed that there an oracle who gives the correct label (also known as gold standard or ground truth) for each instance available in training set. Nevertheless, many real-world problems, instead of standard, we have access to some annotations (possibly noisy) provided by multiple annotators with different unknown levels expertise. Then, not appropriate use trivial methods, i.e., majority voting, estimate actual from due this way assumes homogeneity performance labelers. Here, introduce new kernel alignment-based annotator relevance analysis–(KAAR) approach code expertise averaged matching between input features and expert labels. So, sample predicted convex combination classifiers adopting achieved KAAR-based coding. Experimental results show our methodology can even if available, defeating state-of-the-art techniques.