Document Detail

Testing the Risk of Bias tool showed low reliability between individual reviewers and across consensus assessments of reviewer pairs.
MedLine Citation:
PMID:  22981249     Owner:  NLM     Status:  Publisher    
OBJECTIVES: To assess the reliability of the Cochrane Risk of Bias (ROB) tool between individual raters and across consensus agreements of pairs of reviewers and examine the impact of study-level factors on reliability. STUDY DESIGN AND SETTING: Two reviewers assessed risk of bias for 154 randomized controlled trials (RCTs). For 30 RCTs, two reviewers from each of four centers assessed risk of bias and reached consensus. We assessed interrater agreement using kappas and the impact of study-level factors through subgroup analyses. RESULTS: Reliability between two reviewers was fair for most domains (κ=0.24-0.37), except sequence generation (κ=0.79, substantial). Reliability results across reviewer pairs: sequence generation, moderate (κ=0.60); allocation concealment and "other sources of bias," fair (κ=0.37-0.27); and other domains, slight (κ=0.05-0.09). Reliability was influenced by the nature of the outcome, nature of the intervention, study design, trial hypothesis, and funding source. Variability resulted from different interpretation of the tool rather than different information identified in the study reports. CONCLUSION: Low agreement has implications for interpreting systematic reviews. These findings suggest the need for detailed guidance in assessing the risk of bias.
Lisa Hartling; Michele P Hamm; Andrea Milne; Ben Vandermeer; P Lina Santaguida; Mohammed Ansari; Alexander Tsertsvadze; Susanne Hempel; Paul Shekelle; Donna M Dryden
Related Documents :
20501449 - Enteropathic histopathological features may be associated with shwachman-diamond syndrome.
23646909 - Systematic reviews published in the january 2013 issue of the cochrane library.
16460579 - Septate fungal invasion in masked mastoiditis: a diagnostic dilemma.
16417849 - Reversible posterior leukoencephalopathy syndrome: report of three cases.
2302859 - Occupational traumatic lymphedema of the hands.
20104609 - Holoprosencephaly and ectrodactyly: report of three new patients and review of the lite...
6986009 - Splenic abscess.
16175069 - Remember 'a posteriori diagnosis' of carbon monoxide poisoning.
10602169 - Donor cell leukemia: report of a case occurring 11 years after allogeneic bone marrow t...
Publication Detail:
Type:  JOURNAL ARTICLE     Date:  2012-9-12
Journal Detail:
Title:  Journal of clinical epidemiology     Volume:  -     ISSN:  1878-5921     ISO Abbreviation:  J Clin Epidemiol     Publication Date:  2012 Sep 
Date Detail:
Created Date:  2012-9-17     Completed Date:  -     Revised Date:  -    
Medline Journal Info:
Nlm Unique ID:  8801383     Medline TA:  J Clin Epidemiol     Country:  -    
Other Details:
Languages:  ENG     Pagination:  -     Citation Subset:  -    
Copyright Information:
Copyright © 2012 Elsevier Inc. All rights reserved.
Department of Pediatrics, Alberta Research Centre for Health Evidence and the University of Alberta Evidence-based Practice Center, University of Alberta, 4-472 Edmonton Clinic Health Academy, 11405-87 Avenue, Edmonton, Alberta, Canada T5G 1C9. Electronic address:
Export Citation:
APA/MLA Format     Download EndNote     Download BibTex
MeSH Terms

From MEDLINE®/PubMed®, a database of the U.S. National Library of Medicine

Previous Document:  The withholding of test results as a means of assessing the effectiveness of treatment in test-posit...
Next Document:  Calculating additive treatment effects from multiple randomized trials provides useful estimates of...