Assessing the Validity Evidence of an Objective Structured Assessment Tool of Technical Skills for Neonatal Lumbar Punctures

摘要: Background The lumbar puncture (LP) is a procedural competency deemed necessary by the Accreditation Council for Graduate Medical Education and Emergency Medicine Pediatric Residency Review Committees. The emergency department (ED) primary site residents to be evaluated performing neonatal LPs. Current evaluation methods lack validity evidence as assessment tools. Objectives This was pilot study develop an objective structured of technical skills LP (OSATS-LP) document instrument in regard five sources test validity: content, response process, relation other variables, inter-rater reliability, consequences testing. Methods Pediatric were videotaped fall 2011 comparison faculty resident performance during using video-delayed format. Residents completed demographic experience survey evaluating relations variables. Content process obtained through expert panel meetings resulted following seven domains OSATS-LP: preparation, positioning, analgesia, needle insertion, cerebrospinal fluid (CSF) collection, management laboratory studies, sterility. t-tests assessed significance between level training, previous intensive care unit experience, residents' self-assessed confidence with their total score. agreement OSATS-LP Fleiss' kappa each domain. Results Sixteen pediatric simulation six raters (96 ratings). sterility CSF collection had moderate statistical reliability (κ = 0.41 0.51, respectively). laboratories substantial (κ = 0.60, 0.62, positioning insertion less reliable (κ = 0.16 0.16, Individuals who one or more rotations (NICU) higher score (12.5 vs. 16.9; p < 0.01). own perception ability perform unsupervised did not result score. Conclusions The has reasonable four validity. This serves launching point this tool clinical environments such ED and, therefore, potential provide real-time formative summative feedback improve ultimately lead improvements patient care. Resumen: Comprobacion de la Validez una Herramienta Evaluacion Estructurada Objetiva Habilidades Tecnicas para las Punciones Lumbares Neonatales Introduccion La puncion (PL) es un procedimiento cuya competencia se considera necesaria por el y los El servicio urgencias (SU) lugar primario evaluar residentes en realizacion PL neonatal. Los metodos evaluacion actuales carecen evidencia valida como herramientas evaluacion. Objetivos Este fue estudio piloto desarrollar estructurada objetiva habilidades tecnicas documentar validez instrumento respecto cinco fuentes test: contenido, proceso respuesta, relacion otras fiabilidad interobservador consecuencias del test. Metodologia Los pediatria fueron grabados otono comparacion docente rendimiento residente durante usando formato video diferido. completaron encuesta experiencia demografica mediante relaciones con La contenido respuesta obtuvo traves reuniones expertos resulto siguientes siete dominios preparacion, posicion, insercion aguja, recogida liquido cefalorraquideo (LCR), manejo estudios laboratorio esterilidad. significacion entre nivel entrenamiento, previa unidad cuidados intensivos confianza autoevaluada su puntuacion analizo t Student. concordancia indice Fleiss cada dominio. Resultados Dieciseis simulacion seis evaluadores que examinaron clasificaciones). esterilidad LCR tuvieron moderada (k = 0,41 0,51, respectivamente). analgesia sustancial (k = 0,60, 0,62, posicion e aguja menos fiables (k = 0,16 0,16, habian realizado mas correlacionaron mayor (coeficiente correlacion Pearson = 0,5, p < 0,05). sujetos completado o rotaciones alta (12,5 16,9; p < 0,01). autopercepcion capacidad realizar no supervisada alta. Conclusiones La tiene razonable cuatro test. Este sirve punto partida usar esta herramienta ambientes clinicos SU, ello, potencialidad proporcionar formacion tiempo real retroalimentacion sumativa mejorar residentes, conducir finalmente mejoras atencion paciente.