Liberal relevance criteria of TREC -

摘要: Most test collections (like TREC and CLEF) for experimental research in information retrieval apply binary relevance assessments. This paper introduces a four-point scale reports the findings of project which TREC-7 TREC-8 document pools on 38 topics were reassessed. The goal reassessment was to build subcollection experiments highly relevant documents learn about assessment process as well characteristics multigraded corpus.Relevance criteria defined so that distinction made between rich topical (relevant documents) poor (marginally documents). It turned out 50% assessed regarded marginal. corpus lessons learned from are discussed. need develop more elaborated schemes is emphasized.