Document Detail

The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews.
MedLine Citation:
PMID:  20156912     Owner:  NLM     Status:  MEDLINE    
OBJECTIVE: To examine the prevalence of outcome reporting bias-the selection for publication of a subset of the original recorded outcome variables on the basis of the results-and its impact on Cochrane reviews.
DESIGN: A nine point classification system for missing outcome data in randomised trials was developed and applied to the trials assessed in a large, unselected cohort of Cochrane systematic reviews. Researchers who conducted the trials were contacted and the reason sought for the non-reporting of data. A sensitivity analysis was undertaken to assess the impact of outcome reporting bias on reviews that included a single meta-analysis of the review primary outcome.
RESULTS: More than half (157/283 (55%)) the reviews did not include full data for the review primary outcome of interest from all eligible trials. The median amount of review outcome data missing for any reason was 10%, whereas 50% or more of the potential data were missing in 70 (25%) reviews. It was clear from the publications for 155 (6%) of the 2486 assessable trials that the researchers had measured and analysed the review primary outcome but did not report or only partially reported the results. For reports that did not mention the review primary outcome, our classification regarding the presence of outcome reporting bias was shown to have a sensitivity of 88% (95% CI 65% to 100%) and specificity of 80% (95% CI 69% to 90%) on the basis of responses from 62 trialists. A third of Cochrane reviews (96/283 (34%)) contained at least one trial with high suspicion of outcome reporting bias for the review primary outcome. In a sensitivity analysis undertaken for 81 reviews with a single meta-analysis of the primary outcome of interest, the treatment effect estimate was reduced by 20% or more in 19 (23%). Of the 42 meta-analyses with a statistically significant result only, eight (19%) became non-significant after adjustment for outcome reporting bias and 11 (26%) would have overestimated the treatment effect by 20% or more.
CONCLUSIONS: Outcome reporting bias is an under-recognised problem that affects the conclusions in a substantial proportion of Cochrane reviews. Individuals conducting systematic reviews need to address explicitly the issue of missing outcome data for their review to be considered a reliable source of evidence. Extra care is required during data extraction, reviewers should identify when a trial reports that an outcome was measured but no results were reported or events observed, and contact with trialists should be encouraged.
Jamie J Kirkham; Kerry M Dwan; Douglas G Altman; Carrol Gamble; Susanna Dodd; Rebecca Smyth; Paula R Williamson
Publication Detail:
Type:  Journal Article; Research Support, Non-U.S. Gov't     Date:  2010-02-15
Journal Detail:
Title:  BMJ (Clinical research ed.)     Volume:  340     ISSN:  1756-1833     ISO Abbreviation:  BMJ     Publication Date:  2010  
Date Detail:
Created Date:  2010-02-16     Completed Date:  2010-02-25     Revised Date:  2014-03-17    
Medline Journal Info:
Nlm Unique ID:  8900488     Medline TA:  BMJ     Country:  England    
Other Details:
Languages:  eng     Pagination:  c365     Citation Subset:  AIM; IM    
Export Citation:
APA/MLA Format     Download EndNote     Download BibTex
MeSH Terms
Bias (Epidemiology)*
Cohort Studies
Data Collection
Outcome and Process Assessment (Health Care)
Randomized Controlled Trials as Topic / standards*
Research Design
Review Literature as Topic*
Grant Support
10859//Cancer Research UK; G0800792//Medical Research Council; //Medical Research Council

From MEDLINE®/PubMed®, a database of the U.S. National Library of Medicine

Previous Document:  Prophylactic antibiotics for burns patients: systematic review and meta-analysis.
Next Document:  The financial burden of HIV care, including antiretroviral therapy, on patients in three sites in In...