Inequalities between multi-rater kappas

摘要: The paper presents inequalities between four descriptive statistics that have been used to measure the nominal agreement two or more raters. Each of is a function pairwise information. Light's kappa and Hubert's are multi-rater versions Cohen's kappa. Fleiss' extension Scott's pi, whereas Randolph's generalizes Bennett et al. S multiple While consistent ordering numerical values these measures has frequently observed in practice, there thus far no theoretical proof general inequality among measures. It proved lower bound kappa, an upper if all tables weakly marginal symmetric raters assign certain minimum proportion objects specified category.