| Objectivity in objective structured clinical examinations: checklists are no substitute for examiner commitment. | |
| | |
MedLine Citation:
|
PMID: 12584104 Owner: NLM Status: MEDLINE |
Abstract/OtherAbstract:
|
PURPOSE: This study explored factors that contribute to objectivity in objective structured clinical examinations (OSCEs). The authors quantified the effect of examiners on interrater reliability and separated this effect from that of station construction, determined the effect of objectification on station reliability and validity, and explored examiner factors that may contribute to interrater reliability. METHOD: Data came from examiners' mark sheets from four annual OSCEs (1997-2000). The OSCEs were conducted identically and simultaneously at three sites, within the University of Otago medical school in New Zealand, with two examiners at each station. The contribution to interrater correlations of station construction and mark sheet compared with examiners' contribution was partitioned out using a random-effects analysis of variance. For one OSCE, a multiple linear regression was used to determine the independent contributions to interrater reliability of the number of checklist items per mark sheet, examiner experience, and examiner involvement in station construction. RESULTS: Station construction and mark sheets contributed 10.1% and examiners contributed 89.9% to the variation in interrater reliability. Following multivariate analysis, the number of items per mark sheet was negatively associated, and examiner involvement in station construction was positively associated, with interrater reliability. Examiner experience in examining or in clinical medicine was not associated with interrater reliability. There was a negative, but nonsignificant, correlation between number of items per mark sheet and that station's correlation with the aggregate OSCE mark. CONCLUSIONS: The contribution of objective mark sheets to objectivity is relatively minor compared with examiners' contribution. Increasing the number of checklist items per mark sheet decreased both reliability and validity. Achieving objectivity requires diligent examiners who are involved in the whole assessment. |
| | |
Authors:
|
Tim J Wilkinson; Christopher M Frampton; Mark Thompson-Fawcett; Tony Egan |
Related Documents
:
|
21496954 - A web-based program to ensure compliance of medical staff providers with mandated healt... 17630084 - An innovative advance to increase the use of the vaccine information statement. 21437544 - Bedbugs (heteroptera, cimicidae): an etiology of pruritus to be remembered. 472074 - Follow-up of 11 xyy males with impulsive and/or sex-offending behaviour. 786234 - Dr alexander graham bell--audiologist and speech therapist. 8817974 - Subchronic centrifugal mechanical assist in an unheparinized calf model. |
Publication Detail:
|
Type: Evaluation Studies; Journal Article |
Journal Detail:
|
Title: Academic medicine : journal of the Association of American Medical Colleges Volume: 78 ISSN: 1040-2446 ISO Abbreviation: Acad Med Publication Date: 2003 Feb |
Date Detail:
|
Created Date: 2003-02-13 Completed Date: 2003-03-19 Revised Date: 2004-11-17 |
Medline Journal Info:
|
Nlm Unique ID: 8904605 Medline TA: Acad Med Country: United States |
Other Details:
|
Languages: eng Pagination: 219-23 Citation Subset: AIM; IM |
Affiliation:
|
Christchurch School of Medicine and Health Sciences, University of Otago, Christchurch, New Zealand. tim.wilkinson@chmeds.ac.nz |
Export Citation:
|
APA/MLA Format Download EndNote Download BibTex |
| MeSH Terms | |
Descriptor/Qualifier:
|
Analysis of Variance Education, Medical, Undergraduate* Educational Measurement / methods* Humans New Zealand Regression Analysis Reproducibility of Results |
From MEDLINE®/PubMed®, a database of the U.S. National Library of Medicine
Previous Document: The roles and experiences of nonaffiliated and non-scientist members of institutional review boards.
Next Document: Protecting subjects, preserving trust, promoting progress I: policy and guidelines for the oversight...