Document Detail

Toward robust information: data quality and inter-rater reliability in the American College of Surgeons National Surgical Quality Improvement Program.
MedLine Citation:
PMID:  20123325     Owner:  NLM     Status:  MEDLINE    
BACKGROUND: Data used for evaluating quality of medical care need to be of high reliability to ensure valid quality assessment and benchmarking. The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) has continually emphasized the collection of highly reliable clinical data through its program infrastructure. STUDY DESIGN: We provide a detailed description of the various mechanisms used in ACS NSQIP to assure collection of high quality data, including training of data collectors (surgical clinical reviewers) and ongoing audits of data reliability. For the 2005 through 2008 calendar years, inter-rater reliability was calculated overall and for individual variables using percentages of agreement between the data collector and the auditor. Variables with > 5% disagreement are flagged for educational efforts to improve accurate collection. Cohen's kappa was estimated for selected variables from the 2007 audit year. RESULTS: Inter-rater reliability audits show that overall disagreement rates on variables have fallen from 3.15% in 2005 (the first year of public enrollment in ACS NSQIP) to 1.56% in 2008. In addition, disagreement levels for individual variables have continually improved, with 26 individual variables demonstrating > 5% disagreement in 2005, to only 2 such variables in 2008. Estimated kappa values suggest substantial or almost perfect agreement for most variables. CONCLUSIONS: The ACS NSQIP has implemented training and audit procedures for its hospital participants that are highly effective in collecting robust data. Audit results show that data have been reliable since the program's inception and that reliability has improved every year.
Mira Shiloach; Stanley K Frencher; Janet E Steeger; Katherine S Rowell; Kristine Bartzokis; Majed G Tomeh; Karen E Richards; Clifford Y Ko; Bruce L Hall
Related Documents :
10165115 - Distance education outcomes in clinical laboratory science.
16937045 - Reliability of predictors of study success in medicine.
1448025 - Planning, designing and developing an assessment tool for a 12-week accident emergency ...
17430365 - Psychometric properties of the obsessive compulsive inventory-revised among icelandic c...
16708455 - Pharmaceuticals: pharmaceutical cost controls--2005. end of year issue brief.
3643995 - Antenatal education for cesarean birth: extension of a field test.
Publication Detail:
Type:  Evaluation Studies; Journal Article; Validation Studies     Date:  2009-11-22
Journal Detail:
Title:  Journal of the American College of Surgeons     Volume:  210     ISSN:  1879-1190     ISO Abbreviation:  J. Am. Coll. Surg.     Publication Date:  2010 Jan 
Date Detail:
Created Date:  2010-02-03     Completed Date:  2010-03-08     Revised Date:  -    
Medline Journal Info:
Nlm Unique ID:  9431305     Medline TA:  J Am Coll Surg     Country:  United States    
Other Details:
Languages:  eng     Pagination:  6-16     Citation Subset:  AIM; IM    
Copyright Information:
Copyright (c) 2010 American College of Surgeons. Published by Elsevier Inc. All rights reserved.
Division of Research and Optimal Patient Care, American College of Surgeons, Chicago, IL 60611, USA.
Export Citation:
APA/MLA Format     Download EndNote     Download BibTex
MeSH Terms
Data Collection / standards*
General Surgery / organization & administration*
Hospitals, Veterans / organization & administration*
Medical Audit / methods,  standards
Medical Records / standards*
Policy Making
Program Evaluation
Quality Assurance, Health Care / organization & administration*
Reproducibility of Results
Thoracic Surgery / organization & administration*
United States

From MEDLINE®/PubMed®, a database of the U.S. National Library of Medicine

Previous Document:  Abnormal Liver Function in Relation to Hemodynamic Profile in Heart Failure Patients.
Next Document:  Measuring Communication in the Surgical ICU: Better Communication Equals Better Care.