| Detecting cheating in written medical examinations by statistical analysis of similarity of answers: pilot study. | |
| | |
MedLine Citation:
|
PMID: 15879391 Owner: NLM Status: MEDLINE |
Abstract/OtherAbstract:
|
OBJECTIVE: To assess whether a computer program using a variant of Angoff's method can detect anomalous behaviour indicative of cheating in multiple choice medical examinations. DESIGN: Statistical analysis of 11 examinations held by the Royal College of Paediatrics and Child Health. SETTING: UK postgraduate medical examination. PARTICIPANTS: Examination candidates. MAIN OUTCOME MEASURES: Detection of anomalous candidate pairs by regression of similarity of correct answers in all possible pairs of candidates on the overall proportion of correct answers. Anomalous pairs were subsequently assessed in terms of examination centres and the seating plan of candidates, to assess adjacency. RESULTS: The 11 examinations were taken by a total of 11,518 candidates, and Acinonyx examined 6,178,628 pairs of candidates. Two examinations showed no anomalies, and one examination found an anomaly resulting from a scanning error. The other eight examinations showed 13 anomalies compatible with cheating, and in each pair the two candidates had sat the examination at the same centre, and for six examinations with seating plans, the candidates in the anomalous pairs had been seated side by side. The raw probabilities of the anomalies varied from 3.9x10(-11) to 9.3x10(-30) (median = 1.1x10(-17)), with Bonferroni-corrected probabilities in the range 2.4x10(-5) to 4.1x10(-24) (median = 1.6x10(-11)). This suggests that one anomalous pair is found for every 1000 or so candidates taking this postgraduate examination. CONCLUSIONS: This statistical technique identified a small proportion of candidates who had very similar patterns of correctly answered questions. The likelihood is that one candidate has copied from the other, or that there was collusion, or that a technical error occurred in the exams department (as happened in a single case). Analysis of similarities can be used to identify cheating and as part of the quality assurance process of postgraduate medical examinations. |
| | |
Authors:
|
I C McManus; Tom Lissauer; S E Williams |
Publication Detail:
|
Type: Journal Article |
Journal Detail:
|
Title: BMJ (Clinical research ed.) Volume: 330 ISSN: 1756-1833 ISO Abbreviation: BMJ Publication Date: 2005 May |
Date Detail:
|
Created Date: 2005-05-09 Completed Date: 2005-05-19 Revised Date: 2012-03-06 |
Medline Journal Info:
|
Nlm Unique ID: 8900488 Medline TA: BMJ Country: England |
Other Details:
|
Languages: eng Pagination: 1064-6 Citation Subset: AIM; IM |
Affiliation:
|
Department of Psychology, University College London, London WC1E 6BT. i.mcmanus@ucl.ac.uk |
Export Citation:
|
APA/MLA Format Download EndNote Download BibTex |
| MeSH Terms | |
Descriptor/Qualifier:
|
Deception* Education, Medical, Graduate / statistics & numerical data* Educational Measurement / statistics & numerical data* England Humans Pediatrics / education* Pilot Projects Probability |
| Comments/Corrections | |
Comment In:
|
BMJ. 2005 Jun 25;330(7506):1510; discussion 1510
[PMID:
15976438
]
BMJ. 2005 Jun 25;330(7506):1509-10; discussion 1510 [PMID: 15976432 ] |
From MEDLINE®/PubMed®, a database of the U.S. National Library of Medicine
Previous Document: Effect of combinations of drugs on all cause mortality in patients with ischaemic heart disease: nes...
Next Document: Neurotrophin overexpression in lower airways of infants with respiratory syncytial virus infection.