作者: Kin Wah Fung , William T. Hole , Stuart J. Nelson , Suresh Srinivasan , Tammy Powell
DOI: 10.1197/JAMIA.M1767
关键词: Rank correlation 、 Natural language processing 、 Information retrieval 、 Unified Medical Language System 、 Quality (business) 、 Subject (documents) 、 Artificial intelligence 、 SNOMED CT 、 Inter-rater reliability 、 Vocabulary 、 Computer science 、 Kappa
摘要: Objective: The integration of SNOMED CT into the Unified Medical Language System (UMLS) involved alignment two views synonymy that were different because vocabulary systems have intended purposes and editing principles. UMLS is organized according to one view synonymy, but its structure also represents all individual present in source vocabularies. Despite progress knowledge-based automation development maintenance vocabularies, manual curation still main method determining synonymy. aim this study was investigate quality human judgment Design: Sixty pairs potentially controversial synonyms reviewed by 11 domain experts (six editors five noneditors), scores assigned degree Measurements: each subject compared gold standard (the overall mean score subjects) assess accuracy. Agreement between noneditors measured comparing noneditors. Results: Average accuracy 71% for 75% (difference not statistically significant). Mean showed significant positive correlation (Spearman's rank coefficient 0.654, two-tailed p , 0.01) with a concurrence rate an interrater agreement kappa 0.43. Conclusion: comparable nonediting experts. There reasonable groups.