Document Detail

Testing the Risk of Bias tool showed low reliability between individual reviewers and across consensus assessments of reviewer pairs.
MedLine Citation:
PMID:  22981249     Owner:  NLM     Status:  Publisher    
OBJECTIVES: To assess the reliability of the Cochrane Risk of Bias (ROB) tool between individual raters and across consensus agreements of pairs of reviewers and examine the impact of study-level factors on reliability. STUDY DESIGN AND SETTING: Two reviewers assessed risk of bias for 154 randomized controlled trials (RCTs). For 30 RCTs, two reviewers from each of four centers assessed risk of bias and reached consensus. We assessed interrater agreement using kappas and the impact of study-level factors through subgroup analyses. RESULTS: Reliability between two reviewers was fair for most domains (κ=0.24-0.37), except sequence generation (κ=0.79, substantial). Reliability results across reviewer pairs: sequence generation, moderate (κ=0.60); allocation concealment and "other sources of bias," fair (κ=0.37-0.27); and other domains, slight (κ=0.05-0.09). Reliability was influenced by the nature of the outcome, nature of the intervention, study design, trial hypothesis, and funding source. Variability resulted from different interpretation of the tool rather than different information identified in the study reports. CONCLUSION: Low agreement has implications for interpreting systematic reviews. These findings suggest the need for detailed guidance in assessing the risk of bias.
Lisa Hartling; Michele P Hamm; Andrea Milne; Ben Vandermeer; P Lina Santaguida; Mohammed Ansari; Alexander Tsertsvadze; Susanne Hempel; Paul Shekelle; Donna M Dryden
Related Documents :
2302859 - Occupational traumatic lymphedema of the hands.
2679059 - Amyloidosis and reiter's syndrome: report of a case and review of the literature.
20501449 - Enteropathic histopathological features may be associated with shwachman-diamond syndrome.
19091289 - Serotonin syndrome in a patient taking lexapro and flexeril: a case report.
7205289 - Ischaemic neuropathy of the lumbosacral plexus following intragluteal injection.
10779849 - Kabuki syndrome - report of six cases and review of the literature with emphasis on ocu...
19390029 - Comment on "colossal ionic conductivity at interfaces of epitaxial zro2:y2o3/srtio3 het...
24639949 - Acute simultaneous ruptures of the anterior cruciate ligament and patellar tendon.
21779609 - A review of electrode materials for electrochemical supercapacitors.
Publication Detail:
Type:  JOURNAL ARTICLE     Date:  2012-9-12
Journal Detail:
Title:  Journal of clinical epidemiology     Volume:  -     ISSN:  1878-5921     ISO Abbreviation:  J Clin Epidemiol     Publication Date:  2012 Sep 
Date Detail:
Created Date:  2012-9-17     Completed Date:  -     Revised Date:  -    
Medline Journal Info:
Nlm Unique ID:  8801383     Medline TA:  J Clin Epidemiol     Country:  -    
Other Details:
Languages:  ENG     Pagination:  -     Citation Subset:  -    
Copyright Information:
Copyright © 2012 Elsevier Inc. All rights reserved.
Department of Pediatrics, Alberta Research Centre for Health Evidence and the University of Alberta Evidence-based Practice Center, University of Alberta, 4-472 Edmonton Clinic Health Academy, 11405-87 Avenue, Edmonton, Alberta, Canada T5G 1C9. Electronic address:
Export Citation:
APA/MLA Format     Download EndNote     Download BibTex
MeSH Terms

From MEDLINE®/PubMed®, a database of the U.S. National Library of Medicine

Previous Document:  The withholding of test results as a means of assessing the effectiveness of treatment in test-posit...
Next Document:  Calculating additive treatment effects from multiple randomized trials provides useful estimates of...