Document Detail

Reliability of the peer-review process for adverse event rating.
MedLine Citation:
PMID:  22844445     Owner:  NLM     Status:  MEDLINE    
BACKGROUND: Adverse events are poor patient outcomes caused by medical care. Their identification requires the peer-review of poor outcomes, which may be unreliable. Combining physician ratings might improve the accuracy of adverse event classification.
OBJECTIVE: To evaluate the variation in peer-reviewer ratings of adverse outcomes; determine the impact of this variation on estimates of reviewer accuracy; and determine the number of reviewers who judge an adverse event occurred that is required to ensure that the true probability of an adverse event exceeded 50%, 75% or 95%.
METHODS: Thirty physicians rated 319 case reports giving details of poor patient outcomes following hospital discharge. They rated whether medical management caused the outcome using a six-point ordinal scale. We conducted latent class analyses to estimate the prevalence of adverse events as well as the sensitivity and specificity of each reviewer. We used this model and Bayesian calculations to determine the probability that an adverse event truly occurred to each patient as function of their number of positive ratings.
RESULTS: The overall median score on the 6-point ordinal scale was 3 (IQR 2,4) but the individual rater median score ranged from a minimum of 1 (in four reviewers) to a maximum median score of 5. The overall percentage of cases rated as an adverse event was 39.7% (3798/9570). The median kappa for all pair-wise combinations of the 30 reviewers was 0.26 (IQR 0.16, 0.42; Min = -0.07, Max = 0.62). Reviewer sensitivity and specificity for adverse event classification ranged from 0.06 to 0.93 and 0.50 to 0.98, respectively. The estimated prevalence of adverse events using a latent class model with a common sensitivity and specificity for all reviewers (0.64 and 0.83 respectively) was 47.6%. For patients to have a 95% chance of truly having an adverse event, at least 3 of 3 reviewers are required to deem the outcome an adverse event.
CONCLUSION: Adverse event classification is unreliable. To be certain that a case truly represents an adverse event, there needs to be agreement among multiple reviewers.
Alan J Forster; Monica Taljaard; Carol Bennett; Carl van Walraven
Related Documents :
4038845 - Autoimmune thrombocytopenic purpura in the course of kaposi's sarcoma--report of a case.
8052895 - Simultaneous presentation of thrombotic thrombocytopenic purpura and systemic lupus ery...
15083895 - Henoch-schönlein purpura in two brothers imprisoned in the same jail: presentation two ...
24039125 - Non-immune hydrops fetalis: a prospective study of 53 cases.
12834545 - Sarcomatous transformation of prostatic adenocarcinoma after radio- and chemotherapy.
15096155 - Bilateral naevus of ota: a rare manifestation in a caucasian.
Publication Detail:
Type:  Journal Article; Research Support, Non-U.S. Gov't     Date:  2012-07-26
Journal Detail:
Title:  PloS one     Volume:  7     ISSN:  1932-6203     ISO Abbreviation:  PLoS ONE     Publication Date:  2012  
Date Detail:
Created Date:  2012-07-30     Completed Date:  2012-11-28     Revised Date:  2013-07-12    
Medline Journal Info:
Nlm Unique ID:  101285081     Medline TA:  PLoS One     Country:  United States    
Other Details:
Languages:  eng     Pagination:  e41239     Citation Subset:  IM    
The Ottawa Hospital, Ottawa, Ontario, Canada.
Export Citation:
APA/MLA Format     Download EndNote     Download BibTex
MeSH Terms
Outcome Assessment (Health Care) / methods*
Peer Review / methods*
Grant Support
79977//Canadian Institutes of Health Research

From MEDLINE®/PubMed®, a database of the U.S. National Library of Medicine

Previous Document:  A novel class of anti-HIV agents with multiple copies of enfuvirtide enhances inhibition of viral re...
Next Document:  Protons sensitize epithelial cells to mesenchymal transition.