Document Detail

Bias due to changes in specified outcomes during the systematic review process.
MedLine Citation:
PMID:  20339557     Owner:  NLM     Status:  MEDLINE    
BACKGROUND: Adding, omitting or changing outcomes after a systematic review protocol is published can result in bias because it increases the potential for unacknowledged or post hoc revisions of the planned analyses. The main objective of this study was to look for discrepancies between primary outcomes listed in protocols and in the subsequent completed reviews published on the Cochrane Library. A secondary objective was to quantify the risk of bias in a set of meta-analyses where discrepancies between outcome specifications in protocols and reviews were found.
METHODS AND FINDINGS: New reviews from three consecutive issues of the Cochrane Library were assessed. For each review, the primary outcome(s) listed in the review protocol and the review itself were identified and review authors were contacted to provide reasons for any discrepancies. Over a fifth (64/288, 22%) of protocol/review pairings were found to contain a discrepancy in at least one outcome measure, of which 48 (75%) were attributable to changes in the primary outcome measure. Where lead authors could recall a reason for the discrepancy in the primary outcome, there was found to be potential bias in nearly a third (8/28, 29%) of these reviews, with changes being made after knowledge of the results from individual trials. Only 4(6%) of the 64 reviews with an outcome discrepancy described the reason for the change in the review, with no acknowledgment of the change in any of the eight reviews containing potentially biased discrepancies. Outcomes that were promoted in the review were more likely to be significant than if there was no discrepancy (relative risk 1.66 95% CI (1.10, 2.49), p = 0.02).
CONCLUSION: In a review, making changes after seeing the results for included studies can lead to biased and misleading interpretation if the importance of the outcome (primary or secondary) is changed on the basis of those results. Our assessment showed that reasons for discrepancies with the protocol are not reported in the review, demonstrating an under-recognition of the problem. Complete transparency in the reporting of changes in outcome specification is vital; systematic reviewers should ensure that any legitimate changes to outcome specification are reported with reason in the review.
Jamie J Kirkham; Doug G Altman; Paula R Williamson
Related Documents :
19824937 - Pharmacological management of constipation.
20691617 - Effect of hospital and surgeon volume on patient outcomes following treatment of abdomi...
9117767 - Manuscript reviewing: too long a concealed form of scholarship?
19450337 - Bronchiectasis.
18418027 - Management of factitious disorders: a systematic review.
8549607 - Thyroid carcinoma originating in a teratoma of the ovary.
Publication Detail:
Type:  Journal Article; Research Support, Non-U.S. Gov't     Date:  2010-03-22
Journal Detail:
Title:  PloS one     Volume:  5     ISSN:  1932-6203     ISO Abbreviation:  PLoS ONE     Publication Date:  2010  
Date Detail:
Created Date:  2010-03-26     Completed Date:  2011-01-11     Revised Date:  2014-03-17    
Medline Journal Info:
Nlm Unique ID:  101285081     Medline TA:  PLoS One     Country:  United States    
Other Details:
Languages:  eng     Pagination:  e9810     Citation Subset:  IM    
Export Citation:
APA/MLA Format     Download EndNote     Download BibTex
MeSH Terms
Meta-Analysis as Topic
Periodicals as Topic
Publication Bias
Publishing / standards*
Research Design
Review Literature as Topic*
Treatment Outcome
Grant Support
10859//Cancer Research UK; G0500952//Medical Research Council

From MEDLINE®/PubMed®, a database of the U.S. National Library of Medicine

Previous Document:  Host factors determine anti-GM1 response following oral challenge of chickens with Guillain-Barré s...
Next Document:  Surface-based analysis on shape and fractional anisotropy of white matter tracts in Alzheimer's dise...