Interrater reliability and convergent validity of the American Academy for Cerebral Palsy and Developmental Medicine methodology for conducting systematic reviews

作者: Lesley Wiart , KAT Kolaski , Charlene Butler , Laura Vogtle , Lynne Romeiser Logan

DOI: 10.1111/J.1469-8749.2012.04307.X

关键词:

摘要: Aim  The aim of this study was to evaluate the interrater reliability and convergent validity American Academy for Cerebral Palsy Developmental Medicine’s (AACPDM) methodology conducting systematic reviews (group design studies). Method  Four clinicians independently rated 24 articles level evidence conduct using AACPDM methodology. Study also assessed Effective Public Health Practice Project scale. Raters were randomly assigned one two pairs resolve discrepancies. agreement between individual raters calculated kappa (α=0.05) intraclass correlations (ICCs; α=0.05). Spearman’s rank correlation coefficient relationship raters’ categorization quality categories tools. Results  There acceptable (κ=0.77; p<0.001; ICC=0.90) (κ=0.83; ICC=0.96) ratings. four seven questions (κ=0.53–0.87). ICCs (all raters) category ratings (weak, moderate, strong) indicated good (ICC=0.76). rho a significant positive overall comparisons tools (0.52; p<0.001). Conclusions  rating system has reliability. Evaluation its demonstrated reasonable when compared with similar tool.

参考文章(23)
David Calkins, Bradley S. Marino, Rushika J. Fernandopulle, Health care policy ,(1995)
L Lux, S F Sutton, T S Carey, S West, K N Lohr, N McKoy, V King, Systems to rate the strength of scientific evidence. Evidence report/technology assessment (Summary). pp. 1- 11 ,(2002)
Lynne Romeiser Logan, Robbin R Hickman, Susan R Harris, Carolyn B Heriza, Single‐subject research design: recommendations for levels of evidence and quality rating Developmental Medicine & Child Neurology. ,vol. 50, pp. 99- 103 ,(2008) , 10.1111/J.1469-8749.2007.02005.X
Gordon H. Guyatt, R. Brian Haynes, Roman Z. Jaeschke, Deborah J. Cook, Lee Green, C. David Naylor, Mark C. Wilson, W. Scott Richardson, for the Evidence-Based Medicine Working Group, Users' Guides to the Medical Literature JAMA. ,vol. 284, pp. 1290- 1296 ,(2000) , 10.1001/JAMA.284.10.1290
J Deeks, J Dinnes, R D'Amico, A Sowden, C Sakarovitch, F Song, M Petticrew, D Altman, Evaluating non-randomised intervention studies. Health Technology Assessment. ,vol. 7, pp. 1- 173 ,(2003) , 10.3310/HTA7270
Kilem Li Gwet, Computing inter‐rater reliability and its variance in the presence of high agreement British Journal of Mathematical and Statistical Psychology. ,vol. 61, pp. 29- 48 ,(2008) , 10.1348/000711006X126600
Charlene Butler, Henry Chambers, Murray Goldstein, Susan Harris, Judy Leach, Suzann Campbell, Richard Adams, Johanna Darrah, Evaluating research in developmental disabilities: A conceptual framework for reviewing treatment outcomes Developmental Medicine & Child Neurology. ,vol. 41, pp. 55- 59 ,(1999) , 10.1017/S0012162299000110
Fred K. Hoehler, Bias and prevalence effects on kappa viewed in terms of sensitivity and specificity. Journal of Clinical Epidemiology. ,vol. 53, pp. 499- 503 ,(2000) , 10.1016/S0895-4356(99)00174-2
K. Dickersin, To Reform U.S. Health Care, Start with Systematic Reviews Science. ,vol. 329, pp. 516- 517 ,(2010) , 10.1126/SCIENCE.1189330