Assessing Agreement Among Raters And Identifying Atypical Raters Using A Log-Linear Modeling Approach

摘要: When an outcome is rated by several raters, ensuring consistency across raters increases the reliability of measurement. Tanner and Young (1985) proposed a general class log-linear models to assess agreement among K rating scale with C nominal categories. Their methodology can be used pair-wise three or more raters. Rogel et al. (1996, 1998) extended this work assessing various patterns rater sub-groups size K-1. These test assumption exchangeability. Although parameters from these identify atypical no formal inferential procedures are available. I propose approach that exchangeability rater. The global heterogeneous partial model fit data comparisons made, adjusting p-values for multiple made. parameter constantly involved in statistically significant distinguished. premise that, if there rater, at least one will differ remaining K-1 parameters. illustrated using published intestinal biopsy study six (Rogel al., 1998). Overall Type error power correctly assessed via simulation 5. Bonferroni, Sidak, Holm's Step-down Bonferroni Sidak adjustments control overall error. Being able present, improving ratings directly, influence measurement given sample size. Consequently, informative studies conducted interventions (e.g., behavioral, medicinal) may have positive impact on public's health.