A Study on Comparison of Generalized Kappa Statistics in Agreement Analysis

摘要: Agreement analysis is conducted to assess reliability among rating results performed repeatedly on the same subjects by one or more raters. The kappa statistic commonly used when scales are categorical. simple and weighted statistics measure degree of agreement between two raters, generalized than In this paper, we compare performance four different proposed Fleiss (1971), Conger (1980), Randolph (2005), Gwet (2008a). We also examine how sensitive each can be marginal probability distribution as whether balancedness and/or homogeneity hold not. methods compared in terms relative bias coverage rate through simulation studies various scenarios with numbers subjects, categories. A real data example presented illustrate methods.