| Who reviews the reviewers? Feasibility of using a fictitious manuscript to evaluate peer reviewer performance. | |
| | |
MedLine Citation:
|
PMID: 9737492 Owner: NLM Status: MEDLINE |
Abstract/OtherAbstract:
|
STUDY OBJECTIVE: To determine whether a fictitious manuscript into which purposeful errors were placed could be used as an instrument to evaluate peer reviewer performance. METHODS: An instrument for reviewer evaluation was created in the form of a fictitious manuscript into which deliberate errors were placed in order to develop an approach for the analysis of peer reviewer performance. The manuscript described a double-blind, placebo control study purportedly demonstrating that intravenous propranolol reduced the pain of acute migraine headache. There were 10 major and 13 minor errors placed in the manuscript. The work was distributed to all reviewers of Annals of Emergency Medicine for review. RESULTS: The manuscript was sent to 262 reviewers; 203 (78%) reviews were returned. One-hundred ninety-nine reviewers recommended a disposition for the manuscript: 15 recommended acceptance, 117 rejection, and 67 revision. The 15 who recommended acceptance identified 17.3% (95% confidence interval [CI] 11.3% to 23.4%) of the major and 11.8% (CI 7.3% to 16.3%) of the minor errors. The 117 who recommended rejection identified 39.1 % (CI 36.3% to 41.9%) of the major and 25.2% (CI 23.0% to 27.4%) of the minor errors. The 67 who recommended revision identified 29.6% (CI 26.1% to 33.1%) of the major and 22.0% (CI 19.3% to 24.8%) of the minor errors. The number of errors identified differed significantly across recommended disposition. Sixty-eight percent of the reviewers did not realize that the conclusions of the work were not supported by the results. CONCLUSION: These data suggest that the use of a preconceived manuscript into which purposeful errors are placed may be a viable approach to evaluate reviewer performance. Peer reviewers in this study failed to identify two thirds of the major errors in such a manuscript. |
| | |
Authors:
|
W G Baxt; J F Waeckerle; J A Berlin; M L Callaham |
Publication Detail:
|
Type: Journal Article |
Journal Detail:
|
Title: Annals of emergency medicine Volume: 32 ISSN: 0196-0644 ISO Abbreviation: Ann Emerg Med Publication Date: 1998 Sep |
Date Detail:
|
Created Date: 1998-10-01 Completed Date: 1998-10-01 Revised Date: 2007-11-15 |
Medline Journal Info:
|
Nlm Unique ID: 8002646 Medline TA: Ann Emerg Med Country: UNITED STATES |
Other Details:
|
Languages: eng Pagination: 310-7 Citation Subset: AIM; IM |
Affiliation:
|
Annals of Emergency Medicine, Irving, TX, USA. |
Export Citation:
|
APA/MLA Format Download EndNote Download BibTex |
| MeSH Terms | |
Descriptor/Qualifier:
|
Confidence Intervals Confounding Factors (Epidemiology) Data Interpretation, Statistical Double-Blind Method Evaluation Studies as Topic Feasibility Studies Humans Manuscripts, Medical Migraine Disorders / drug therapy Pain Measurement Patient Selection Peer Review, Research / standards* Placebos Propranolol / therapeutic use Publishing / standards* Randomized Controlled Trials as Topic / standards Research Design / standards Vasodilator Agents / therapeutic use |
| Chemical | |
Reg. No./Substance:
|
0/Placebos; 0/Vasodilator Agents; 525-66-6/Propranolol |
| Comments/Corrections | |
Comment In:
|
Ann Emerg Med. 1999 Mar;33(3):356
[PMID:
10036387
]
|
From MEDLINE®/PubMed®, a database of the U.S. National Library of Medicine
Previous Document: Accuracy of reagent strips in detecting hypoglycemia in the emergency department.
Next Document: Effect of attendance at a training session on peer reviewer quality and performance.