| Interevaluator Reliability of a Mock Paramedic Practical Examination. | |
| | |
MedLine Citation:
|
PMID: 22229924 Owner: NLM Status: Publisher |
Abstract/OtherAbstract:
|
Abstract Introduction. Prior to graduation, paramedic students must be assessed for terminal competency and preparedness for national credentialing examinations. Although the procedures for determining competency vary, many academic programs use a practical and/or oral examination, often scored using skill sheets, for evaluating psychomotor skills. However, even with validated testing instruments, the interevaluator reliability of this process is unknown. Objective. We sought to estimate the interevaluator reliability of a subset of paramedic skills as commonly applied in terminal competency testing. Methods. A mock examinee was videotaped performing staged examinations mimicking adult ventilatory management, oral board, and static and dynamic cardiac stations during which the examinee committed a series of prespecified errors. The videotaped performances were then evaluated by a group of qualified evaluators using standardized skill sheets. Interevaluator variability was measured by standard deviation and range, and reliability was evaluated using Krippendorff's alpha. Correlation between scores and evaluator demographics was assessed by Pearson correlation. Results. Total scores and critical errors varied considerably across all evaluators and stations. The mean (± standard deviation) scores were 24.77 (±2.37) out of a possible 27 points for the adult ventilatory management station, 11.69 (±2.71) out of a possible 15 points for the oral board station, 7.79 (±3.05) out of a possible 12 points for the static cardiology station, and 22.08 (±1.46) out of a possible 24 points for the dynamic cardiology station. Scores ranged from 18 to 27 for adult ventilatory management, 7 to 15 for the oral board, 2 to 12 for static cardiology, and 19 to 24 for dynamic cardiology. Krippendorff's alpha coefficients were 0.30 for adult ventilatory management, 0.01 for the oral board, 0.10 for static cardiology, and 0.48 for dynamic cardiology. Critical criteria errors were assigned by 10 (38.5%) evaluators for adult ventilatory management, five (19.2%) for the oral board, and nine (34.6%) for dynamic cardiology. Total scores were not correlated with evaluator demographics. Conclusions. There was high variability and low reliability among qualified evaluators using skill sheets as a scoring tool in the evaluation of a mock terminal competency assessment. Further research is needed to determine the true overall interevaluator reliability of this commonly used approach, as well as the ideal number, training, and characteristics of prospective evaluators. |
| | |
Authors:
|
Melisa Martin; Michael W Hubble; Marianne Hollis; Michael E Richards |
Related Documents
:
|
17434704 - Moving innovations into treatment: a stage-based approach to program change. 10800864 - Do consumers who have a choice of treatment have better outcomes? 16602404 - Treatment outcomes in an integrated civilian and prison mdr-tb treatment program in rus... 20925684 - Teaching community program clinicians motivational interviewing using expert and train-... 16889314 - Changes in body weight and fat mass of men and women in the first year of college: a st... 16494724 - Economics of neuraminidase inhibitor stock piling for pandemic influenza, singapore. |
Publication Detail:
|
Type: JOURNAL ARTICLE Date: 2012-1-9 |
Journal Detail:
|
Title: Prehospital emergency care : official journal of the National Association of EMS Physicians and the National Association of State EMS Directors Volume: - ISSN: 1545-0066 ISO Abbreviation: - Publication Date: 2012 Jan |
Date Detail:
|
Created Date: 2012-1-10 Completed Date: - Revised Date: - |
Medline Journal Info:
|
Nlm Unique ID: 9703530 Medline TA: Prehosp Emerg Care Country: - |
Other Details:
|
Languages: ENG Pagination: - Citation Subset: - |
Affiliation:
|
From the Emergency Medical Care Program (MM, MWH) and the Master of Health Sciences Program (MH), Western Carolina University , Cullowhee , North Carolina ; and the Department of Emergency Medicine, University of New Mexico (MER) , Albuquerque , New Mexico . |
Export Citation:
|
APA/MLA Format Download EndNote Download BibTex |
| MeSH Terms | |
Descriptor/Qualifier:
|
|
From MEDLINE®/PubMed®, a database of the U.S. National Library of Medicine
Previous Document: Dupuytren's Disease.
Next Document: Quorum-sensing non-coding small RNAs use unique pairing regions to differentially control mRNA targe...