Ways of Evaluation of the Annotators in Building the Prague Czech-English Dependency Treebank

摘要: In this paper, we present several ways to measure and evaluate the annotation annotators, proposed used during building of Czech part Prague Czech-English Dependency Treebank. At first, basic principles treebank project are introduced (division three layers: morphological, analytical tectogrammatical). The main paper describes in detail one important phases process: evaluation annotators - inter-annotator agreement, error rate performance. measuring agreement is complicated by fact that data contain added deleted nodes, making alignment between annotations non-trivial. measured a set automatic checking procedures guard validity some invariants data. performance booking web application. All measures later compared related each other.