SemEval-2012 Task 4: Evaluating Chinese Word Similarity

摘要: This task focuses on evaluating word similarity computation in Chinese. We follow the way of Finkelstein et al. (2002) to select pairs. Then we organize twenty undergraduates who are major Chinese linguistics annotate data. Each pair is assigned a score by each annotator. rank pairs average value similar scores among annotators. data used as gold standard. Four systems participating this return their results. evaluate results standard term Kendall's tau value, and show three them have positive correlation with manually created while taus' very small.