Document Detail


Validity and interrater reliability of a regional mock oral board examination.
MedLine Citation:
PMID:  23618452     Owner:  NLM     Status:  In-Data-Review    
Abstract/OtherAbstract:
BACKGROUND: There have been decreasing pass rates recently on the American Board of Surgery Certifying Examination (ABSCE). General surgery residents from the University of Pittsburgh Medical Center, the West Penn Allegheny Health System, the Conemaugh Health System, and Mercy Hospital participate in a mock oral board examination, which is similar to the ABSCE. The aims of the study are to compare examinee performance on the mock oral boards with the ABSCE and to evaluate the interrater reliability of examiner pairs.
METHODS: In this retrospective study from 2003 to 2010, outcomes on the mock oral boards and the first attempt of the ABSCE for chief residents were compared for the 4 regional residency programs. Interrater reliability for examiner pairs was evaluated with agreement and kappa statistics. Nonparametric statistics were performed, with α = 0.05.
RESULTS: A total of 32 of 38 (84.2%) chief residents passed the mock oral boards. The median score for each of the 3 rooms was 6 (clear pass). A total of 37 of 38 (97.4%) residents passed the ABSCE. The sensitivity of the mock oral boards was 83.8%, with a positive predictive value of 96.9%, and an accuracy of 81.6%. A total of 25 of 47 (53.2%) examiner pairs were from the same residency institution, whereas 22 of 47 (46.8%) were from different institutions. The median agreement was 100% (interquartile range (IQR) [100% - 100%]). The median kappa statistic was 1.00 (IQR [0.38-1.00]). The Mann-Whitney U tests showed no difference in agreement or kappa for examiner pairs from the same or from different institutions (p> 0.05).
CONCLUSIONS: The mock oral boards have substantial sensitivity and positive predictive value in relation to the ABSCE. There are also very high levels of interrater agreement and interrater reliability. This regional mock oral board examination is valuable for ABSCE preparation.
Authors:
John L Falcone; Daniel J Gagne; Kenneth K W Lee; Giselle G Hamad
Publication Detail:
Type:  Journal Article    
Journal Detail:
Title:  Journal of surgical education     Volume:  70     ISSN:  1878-7452     ISO Abbreviation:  J Surg Educ     Publication Date:    2013 May-Jun
Date Detail:
Created Date:  2013-04-26     Completed Date:  -     Revised Date:  -    
Medline Journal Info:
Nlm Unique ID:  101303204     Medline TA:  J Surg Educ     Country:  United States    
Other Details:
Languages:  eng     Pagination:  402-7     Citation Subset:  IM    
Copyright Information:
Copyright © 2013. Published by Elsevier Inc.
Affiliation:
Department of Surgery, University of Pittsburgh School of Medicine, University of Pittsburgh Medical Center, Pittsburgh, Pennsylvania. Electronic address: falconej@upmc.edu.
Export Citation:
APA/MLA Format     Download EndNote     Download BibTex
MeSH Terms
Descriptor/Qualifier:

From MEDLINE®/PubMed®, a database of the U.S. National Library of Medicine


Previous Document:  The virtual-patient pilot: testing a new tool for undergraduate surgical education and assessment.
Next Document:  A literature review of professionalism in surgical education: suggested components for development o...