Are VHA administrative location codes valid indicators of specialty substance use disorder treatment?
Article Type: Report
Subject: Substance abuse (Care and treatment)
Substance abuse (Diagnosis)
Substance abuse (Research)
Substance abusers (Health aspects)
Medical care (Quality management)
Medical care (Usage)
Authors: Harris, Alex H.S.
Reeder, Rachelle N.
Ellerbe, Laura
Bowe, Thomas
Pub Date: 12/30/2010
Publication: Name: Journal of Rehabilitation Research & Development Publisher: Department of Veterans Affairs Audience: Academic Format: Magazine/Journal Subject: Health Copyright: COPYRIGHT 2010 Department of Veterans Affairs ISSN: 0748-7711
Issue: Date: Dec 30, 2010 Source Volume: 47 Source Issue: 8
Topic: Event Code: 310 Science & research
Geographic: Geographic Scope: United States Geographic Code: 1USA United States
Accession Number: 246450398
Full Text: INTRODUCTION

The ability to accurately identify the occurrence of specific healthcare events is central to many quality improvement and research efforts. Does a patient with diabetes receive an annual foot exam [1]? Is the patient in the emergency room with pneumonia given antibiotics within 3 hours [2]? Does the patient receive a certain number of outpatient mental health visits after discharge from an inpatient mental health setting [3]? When the scope of these inquiries involves hundreds of thousands of patients, as in national quality monitoring efforts, it is important to operationalize the specified care in a way that minimizes labor intensive strategies, such as chart review, and maximizes the use of preexisting and easily accessible administrative data.

Care identification strategies based on commonly available administrative data are inexpensive and feasible but usually of unknown validity. In this context, validity is measured as the association between the identification of care with a particular strategy (e.g., diagnosis and procedure code combinations) and an often more difficult to obtain "gold standard," such as the direct observation of care or chart review. The purpose of this study was to determine the validity of a substance use disorder (SUD) treatment quality care-identification strategy used by the Veterans Health Administration (VHA) and many researchers studying VHA SUD treatment. To achieve this aim, we examined the clinical progress notes of healthcare encounters tagged with specialty codes (i.e., clinic stop and bed section codes) that are often assumed to signify the provision of SUD treatment.

BACKGROUND

In some cases, the mapping of administrative healthcare data to specific healthcare events is relatively easy. If a patient received a particular medication, procedure, or device that is described by specific codes, a researcher or quality manager may be able to reliably identify the care of interest [4]. For example, a total knee replacement surgery is easily identified by specific surgical procedure codes.

However, in other areas of healthcare, the codes in administrative data may be too general to be useful. For example, mental health care procedure codes describe events such as "individual psychotherapy," "group counseling," and "supportive verbal psychotherapy," yet do not specify the type or target of care. Consequently, these codes are too generic, for example, to accurately identify treatment for SUDs as distinct from care for other mental health conditions, such as depression, especially when both diagnoses are attached to the record.

At the VHA, researchers and quality managers often use outpatient clinic stop codes (Decision Support System Identifiers) and inpatient bed section codes as hybrids of location and specialty treatment codes. However, these codes do not necessarily signify the provision of specialty care [5]. Administratively, "clinics" are established with specific outpatient clinic stop or inpatient bed section (treating specialty) codes. For example, all records generated by "Addiction Clinic A" are automatically tagged with the clinic stop codes determined during the administrative clinic setup process, even for encounters that are not addiction focused (e.g., general mental health or other activities that generate encounter records). With the existing Veterans Health Information Systems and Technology Architecture (VistA) scheduling package, the patient is scheduled into a clinic with predetermined codes and a more appropriate clinic might not be selected upon review of the encounter.

In a broad evaluation of the agreement between administrative data and medical records, Kashner found clinic stop codes to be only moderate indicators of the treatment focus or clinic specialty in VHA outpatient visits and bed section codes to be generally better indicators of ward specialty [6]. However, agreement between clinic stop and bed section codes and the focus of care by specialty varied substantially. The conclusion highlights the importance of validating the assumption that VHA clinic stop and bed section codes signify the provision of specialty care for each specific research and quality measurement application [6].

In monitoring the quality and utilization of SUD treatment, the VHA uses clinic stop and bed section codes to identify and count records that are presumed to include the provision of SUD treatment. For example, the VHA Office of Quality and Performance has developed and monitors the SUD Continuity of Care performance measure, which estimates the proportion of new SUD specialty clinic patients at each facility who receive a certain intensity and duration of SUD treatment. Outpatients meet the performance measure if they have at least two specialty SUD care contacts in each of three successive 30-day periods after initiating a new episode of care. Patients from SUD inpatient or residential treatment programs meet the performance measure if they have at least two specialty SUD outpatient contacts in each of three successive 30-day periods following discharge. Each year, performance targets are set for this and other performance measures and executive compensation is partially contingent on meeting these thresholds [7]. Facilities falling below the threshold are often targeted for remediation and quality improvement efforts.

The specifications of this performance measure rely on the unchecked assumption that records coded with SUD clinic stop or bed section codes reflect the provision of SUD care as opposed to care for other disorders (e.g., smoking, pathological gambling, posttraumatic stress disorder [PTSD]) or other activities (e.g., scheduling appointments, documenting no-shows, making referrals to other programs). In this study, we sought to validate presumed links between SUD specialty treatment codes and receipt of SUD care as documented in clinical progress notes. We also investigated whether rates of association between the SUD specialty treatment codes and documented SUD treatment varied by setting (outpatient, inpatient) or facility and whether the validity of the specialty codes as indicators of treatment might be improved by supplemental information, such as diagnosis and procedure codes.

METHODS

Data Source and Sampling

The data sources for this study were the fiscal year (FY) 2005 VHA National Patient Care Database (NPCD) Event and Bed Section files, which contain records of every healthcare encounter for more than 5 million veterans who annually receive care from VHA, and VistAWeb, which is an intranet Web application of VistA [8]. VistAWeb has a graphical user interface that contains similar information to that found in the more familiar Computerized Patient Record System and allows national chart review. In FY05, roughly 120,000 unique patients, who had more than 2,300,000 encounters with SUD specialty clinic stops or bed section codes, were in the system. Patients seen in SUD specialty settings in FY05 represent about one-third of the 342,753 unique VHA patients with a SUD diagnosis plus 8,571 patients without a SUD diagnosis.

Records were randomly sampled from the NPCD Event (outpatient) and Bed Section (inpatient and residential care) files stratified by four record types: (1) outpatient records that included a SUD clinic stop code and a SUD diagnosis/common procedural terminology (CPT) code combination (n = 700), (2) outpatient records that included a SUD clinic stop code but not a SUD diagnosis/CPT code combination (n = 1,250) (these were oversampled to provide a more reliable estimate of the subtypes [422 with a SUD diagnosis but no mental health CPT code, 445 with a mental health CPT code but no SUD diagnosis, and 383 with neither]), (3) inpatient/residential records that included a SUD bed section code and SUD diagnosis code (n = 700), and (4) inpatient/residential records that included a SUD bed section code but not a SUD diagnosis code (n = 39; all available). For each record type, a chronological list of all records meeting the criteria was constructed, a numeric vector of the same length was randomly generated, and the records with the highest random numbers were selected to produce the desired sample size. Although the sampling strategy allowed for multiple records per patient, only seven patients contributed two records each.

The SUD clinic stop and bed section codes examined in this study were those used in the specifications of the SUD Continuity of Care performance measure (Figure). Although credit clinic stops are considered by the SUD Continuity of Care performance measure, they are not included in the NPCD Event file and were not examined in this study. Some extant SUD codes were not included in the quality measure specifications or the specification for this study because they are no longer in use, are not sufficiently targeted to satisfy the intent of the measure (e.g., SUD compensated work therapy), or involve care that is the target of other performance measures (e.g., 523-opioid substitution/methadone visit). Note that the inpatient/residential category is dominated by admissions to nonacute residential rehabilitation programs.

To address whether the supplemental use of diagnosis codes might increase the validity of clinic stop and bed section codes as indicators of SUD treatment, we sampled both outpatient and inpatient records that had a specialty treatment code and either did or did not have an alcohol or drug use disorder, excluding tobacco use disorders (Figure). We also sampled outpatient records with SUD clinic stop codes that did or did not have relevant mental health CPT codes (Figure). The list of codes was adapted from the procedure codes used in the Healthcare Effectiveness Data and Information Set (HEDIS) SUD Initiation and Engagement measure [9].

Progress Note Extraction

After randomly selecting healthcare records from the NPCD that met our specifications, we extracted the clinical progress notes from VistAWeb. Although our interest was in extracting the progress note for the specific record that included the specialty code, no method exists for precisely matching records located in VHA administrative data to a specific progress note. Although this process was usually straightforward, it was difficult in some cases to determine which of several progress notes was the target on a particular day. In these cases, we extracted all of the progress notes on the day of interest. The progress notes were then entered into a secured database to enable coding and analysis. All identifying information was removed from the extracted progress notes.

Missing Progress Notes

For a surprising number of outpatient records (280 of 1,950 sampled; 14.4%), no progress notes were found on the day of the selected record. Progress notes are supposed to be written on the day of the encounter. It is possible to write the note later and attach the note to the day of the encounter in the record. The analytic treatment of the missing progress notes has no wholly satisfying solution. Removing these records from the denominator seems justified as we had no legitimate means in these cases to judge whether SUD treatment was provided or not. However, this strategy may bias estimates of treatment provision up or down depending on the unknown rate of treatment provision in these encounters. This strategy provides an estimate of the rate of SUD treatment provision in records selected with various administrative codes under the assumption that the rate of SUD care in the records with missing notes is similar to records with observed progress notes. Another strategy would be to retain these records in the denominator and assume that no SUD care was provided. This would produce a lower bound to the question addressed by the first method and directly addresses a subtly different question: What proportion of records selected with various administrative codes contains chart documentation of SUD treatment? We present the results using both methods, but discuss and elaborate on the former method as the primary analysis. We also examined possible reasons why notes may have been missing on the target day and describe our efforts to find them, as well as present data on the range of facility-level rates of missing notes.

Content Analysis and Ratings of Progress Notes

The content analysis and rating procedure was developed following the guidance provided by Stemler [10] and McTavish and Pirro [11] and relied on a selective reduction process, focusing on key words related to SUD treatment (e.g., relapse, recovery, rehabilitation, sobriety, Alcoholics Anonymous, addiction, sponsor, alcohol, cocaine, heroin, naltrexone, disulfiram, antabuse, dual diagnosis, Alcohol Use Disorders Identification Test, Substance Abuse Treatment Program, CAGE). Although key words were useful, they were insufficient for determining the provision of SUD care. Certain contextual factors disqualified a key word occurrence from SUD care designation. For example, the following statement would be classified as SUD care: "Patient needs rehabilitation; Made referral to substance abuse treatment program." Brief assessment of need and referral to specialized care was considered SUD treatment. However, the following statement would not be considered adequate evidence of SUD care: "Patient may need rehabilitation." The term rehabilitation is not specific to SUD treatment. Even if this note said "may need substance abuse rehabilitation," it would not meet our criteria, since this statement would need to be accompanied by a documented referral for or additional provision of SUD treatment. Extensive assessment interviews, such as the Addiction Severity Index, were counted as SUD care because these usually occur as part of the treatment planning and monitoring process rather than the need assessment process. We developed these guidelines to account for the numerous contextual factors inherent in clinical progress notes. In addition, certain headings in the progress notes proved useful in determining the provision of SUD care. For example, the "chief complaint," "presenting problem," "reason for admission," "admitting diagnosis," and "assessment/ plan" headings helped pinpoint whether SUD care was provided during an encounter.

Using this system, two raters independently classified each of the selected records as documenting or not documenting the provision of SUD care. Every 300 notes, the raters compared classifications and resolved discrepancies with the help of a third independent rater. Further, a sample of notes for which agreement existed between the two raters was rated by a third rater as a validation and process quality check. Initial interrater reliability exceeded 85 percent, and final interrater agreement was 100 percent.

Records were further coded by type of treatment. The treatment categories were developed through an iterative process of rating and sorting a pilot sample of records. The categories for records with documentation of SUD care were (1) admission/discharge note from an inpatient/residential stay with documented SUD treatment, (2) SUD outpatient care, (3) detoxification, (4) SUD assessment, and (5) outpatient care partially related to SUD. (Table 1 gives examples of records that were classified in these categories.) Although this report focused on the dichotomous provision of SUD care (yes/no) rather than the type of care provided, further details regarding the distribution of records into these SUD care categories are available.

RESULTS

Table 2 presents the concordance rates (95% confidence interval) between chart review and administrative codes as well as the range of concordance rates and rates of missing notes by Veterans Integrated Service Network (VISN). About 47 percent of the 2,289,922 outpatient records with SUD clinic stops also had a SUD diagnosis code and a relevant mental health CPT code. Of the 700 randomly selected records that met these criteria, 99 (14.1%) had no progress note on the day of care and 553 were found by chart review to have evidence of SUD treatment (92% of those with progress notes, 79% of all records). We found that among records with progress notes, the concordance rate for those with a primary SUD diagnosis (92%) was not significantly higher than for those with a SUD diagnosis that was nonprimary (91%). The specific diagnosis or CPT code connected with the visit did not affect the association with the chart review determination of SUD care. The concordance rate did not vary according to whether the relevant mental health CPT code was primary or nonprimary. In the 48 progress notes without evidence of SUD care, the most common types of care documented were non-SUD mental health treatment, smoking cessation, and other medical care (Table 3). Concordance rates between administrative data and chart review determination of SUD treatment varied substantially by facility (VISN), ranging from 82 to 100 percent.

About 53 percent of the 2,289,922 outpatient records with SUD clinic stops did not have a SUD diagnosis code and a relevant mental health CPT code. The 1,250 randomly sampled records of this type could be further divided into three subtypes having (1) a SUD diagnosis but no relevant mental health CPT code (40% of all records with a SUD clinic stop, 82.5% concordance with chart review); (2) a relevant mental health CPT code but no SUD diagnosis (8% of all records with a SUD clinic stop, 21.9% concordance with chart review); and (3) neither a SUD diagnosis nor a relevant mental health CPT code (5% of all records with a SUD clinic stop, 55.3% concordance with chart review). Overall, of the 1,250 randomly selected records that met these criteria, 181 (14.5%) records had no progress note on the day of care. The types of care documented in 504 progress notes without evidence of SUD care are presented in Table 3, with nonSUD mental health treatment, smoking cessation, and informational and scheduling telephone calls (not telephone care) or letters being the most common. When the primary diagnosis was tobacco use disorder (International Classification of Diseases-9th Revision, Clinical Modification code 305.1), the encounters almost never included the provision of other SUD care (only 1 of 57 records).

Of the 700 inpatient records accompanied by a SUD diagnosis, only one lacked associated chart documentation and 692 of the remaining 699 (99%) were found to have chart review evidence of SUD care. The concordance rate was no different for records with a primary versus nonprimary SUD diagnosis.

Very few inpatient records were found that had a SUD bed section code but no SUD diagnosis (n = 39). However, none of these 39 records had chart review evidence of SUD care. Chart review of these records indicated admission/treatment for pathological gambling (n = 8), other psychiatric disorders (e.g., PTSD, bipolar; n = 20), and other problems (e.g., homelessness, health maintenance; n = 11). Half of these records were from one network, suggesting that an inpatient unit with a primary SUD focus that also accepts patients with other psychiatric disorders is present.

FURTHER EXAMINATION OF "MISSING" PROGRESS NOTES

To investigate whether the missing notes may have been written but attached to the wrong day, we examined the progress notes (if any) on the day before and 2 days after the target day. In 48 percent of the cases, no progress notes were found on the day before or 2 days after the target day. In 28 percent of the cases, notes were found but none that appeared to be plausibly related to the record of interest. In the remaining 24 percent, we found a note that may have been the note of interest attached to the wrong day, but we could not verify that it was in reference to the selected record.

DISCUSSION

These results provide validity evidence and caveats to VHA researchers and quality managers who might use SUD specialty treatment codes as indicators of SUD specialty care. This research also provides an example for researchers in other areas of healthcare who want to better understand the link between VHA clinic stop and bed section codes and the provision of specialty care.

Missing Progress Notes

The fact that more than 14 percent of the outpatient records sampled for this study did not have an associated progress note on the day of service raises important questions about data quality and the documentation of clinical care. No system or code exists for matching records in administrative data to specific progress notes. This problem is exacerbated when progress notes are written in the days after the encounter and not correctly attached to the day of service. Our examination of this issue suggests that perhaps 24 percent of these encounters with missing notes may have been documented with progress notes on another day, but determining which note, if any, mapped onto which record was not straightforward. The other 76 percent of the missing notes (11% of all outpatient notes sampled) appeared to be completely missing. VISNs varied substantially in the extent to which progress notes appeared to be missing on the day of service. Some VISNs had virtually no missing notes, while others exceeded 20 percent of sampled records. As a data quality problem that affects clinical care, quality measurement, and research, this issue should be further investigated and remedied.

Use of Bed Section Codes as Indicators of SUD Care

For inpatient records, SUD bed section codes are almost always paired with SUD diagnoses and chart documentation of SUD treatment. Therefore, researchers and quality managers can reasonably assume that SUD bed section codes reflect care that includes SUD treatment. A very small number of records with SUD bed section codes were not accompanied by a SUD diagnosis. None of these included chart documentation of SUD. This appears to occur primarily in one VISN where patients with other psychiatric disorders (e.g., compulsive gambling, PTSD) are treated within a unit with a SUD focus, perhaps because more appropriate and specific inpatient units are not available. The prevalence of these records is so low (only 39 in an entire year) that they could be included or excluded from most research or quality measurement endeavors without consequence.

Use of Clinic Stop Codes as Indicators of SUD Care

About 47 percent of all outpatient records with SUD clinic stop codes also had a SUD diagnosis/CPT combination and 92 percent (VISN range of 82%-100%) of these records with an associated progress note had evidence of SUD care. However, of the records with an associated progress note that had a SUD clinic stop code and SUD diagnosis but no relevant CPT code, 82.5 percent (VISN range of 57%-100%) had evidence of SUD care. Although one might deem these overall rates of concordance to be adequate for most applications, the VISN ranges of concordance have important implications that are likely to be exacerbated at the facility level, where greater variability is typically observed. In studies of quality measurement applications that involve describing or comparing facilities, it is important to know and account for the varying facility-level concordance rates. For example, if two facilities have vastly different underlying rates of concordance between clinic stop codes and chart documentation of care, it is very difficult to make meaningful comparisons regarding their performance on quality measures that use these codes as indicators of SUD care, such as the SUD Continuity of Care performance measure. Therefore, in facilities that have a low concordance between clinic stop codes and chart documentation of care, efforts should be made to better understand the problems and remedy them.

Chart evidence of SUD care fell to 55.3 percent in records with SUD clinic stops but neither a SUD diagnosis nor a mental health CPT code and to 21.9 percent in those with a mental health CPT code but no SUD diagnosis. Records with mental health CPT codes but not a SUD diagnosis most often involved other types of care occurring in SUD specialty clinics, such as treatment of smoking and gambling disorders.

From a practical standpoint, these later results raise the question of whether researchers and quality managers should use records with SUD clinic stops as indicators of SUD treatment if they lack a SUD diagnosis. Although chart review evidence of SUD care was low in these records (55% without and 22% with a CPT code), these records only account for roughly 13 percent of all records with a SUD clinic stop code. So what difference will culling these less reliable records have on one's accounting system? We plan to examine this question by conducting analyses in which we require the definition of a SUD visit to include both a SUD clinic stop and SUD diagnosis code and determine whether and how much change is observed in the overall rates and rank ordering of the SUD Continuity of Care performance measure, as well as the measure's association with outcomes.

Understanding how administrative records of healthcare encounters are tagged with clinic stop codes demystifies some of the discordant results. As stated previously, administratively "clinics" are established and mapped with specific clinic stop codes, so all records generated by "Clinic A" are tagged with the preset clinic stop codes, regardless of the nature of the encounters. If a "clinic grid" is out of date or does not reflect the varied nature of the work performed by the clinic, stop codes may be generated that do not precisely describe the nature of the care provided. For example a methadone clinic (clinic stop code 523-opioid substitution/methadone visit) may occasionally see patients without opioid use disorders for assessment or for treatment of other SUDs. These encounters are tagged with the 523 stop code even though no opioid substitution therapy was provided. A researcher or quality manager who assumes that all encounters with a 523 clinic stop involve opioid substitution might risk overcounting the provision of that service. This may have become a more significant issue given recent initiatives, such as Advanced Clinic Access, that promote clinical cross-coverage to reduce waiting lists and improve access. Under this initiative, patients are often scheduled into the first available clinic (e.g., mental health) rather than the one that specializes in their primary problem (e.g., alcohol dependence). For these reasons, CPT codes and/or diagnostic codes provide important information to confirm the type of care/services provided during an encounter.

Beyond the implications for those using VHA data, this study highlights the importance of validating assumptions regarding the use of administrative codes as indicators of treatment provision. Other quality measurement systems, such as HEDIS, commonly rely on combinations of diagnosis and procedure codes to identify the provision of specialty care [9]. The validity of these strategies may vary from system to system and between facilities within systems and is rarely checked in any event. Although not without limitations, this study may act as a template for other similar validation studies.

Limitations

Although the progress notes in VistAWeb are very useful for determining the nature of the healthcare encounter, they are not without limitations. Studies attempting to validate the use of progress notes as indicators of what transpired during a healthcare encounter have found generally moderate and variable concordance with direct observation (k ranging from 0.12 to 0.89) [12] and patient surveys (k ranging from 0.1 to 0.9) [13]. Another study found events reported by a standardized patient were often not reported in the medical record and events not reported by a standardized patient were sometimes documented in the record [14-15].

Another limitation of using progress notes as the gold standard for determination of SUD care is that variability exists in the level of detail used to describe outpatient visits. For example, some progress notes contained an abundance of key information for determining SUD care, whereas other progress notes lacked enough detail to make the determination with absolute confidence. Since it was necessary to look at key words (e.g., recovery, addiction) in context, notes lacking detailed information were sometimes difficult to code. In these cases, some arbitration was necessary to carefully consider the note-specific contextual factors and the medical language used. Therefore, though not always entirely satisfying in some cases, consensus among the three raters was the most appropriate way to resolve these issues. Also, given our methodology we could not assess the prevalence of SUD care that occurred in the absence of clinic stop and bed section codes. Also, we used Level I CPT codes in our sampling definitions, but not "H" codes, which are appropriate for use by the many clinical staff in SUD programs who are not licensed independent providers and are not eligible to use Level I CPT codes. Future validation studies should examine the prevalence of the use of these codes and whether they might be used to improve the identification of SUD treatment. Finally, although a specific encounter may not reflect SUD-specific care, it does not mean the patient never received care. However, this study focused on an encounter-level analysis to determine which visits should be counted as SUD care in research and quality measurement applications.

CONCLUSIONS

SUD bed section codes are very specific indicators of the provision of SUD treatment however, other SUD treatment may occur in different bed sections and the methods of this study cannot determine the sensitivity of bed section codes to detect all documented SUD care (e.g., if it occurred in acute psychiatry bed sections). Among the records with SUD clinic stops accompanied by SUD diagnoses and relevant CPT codes, 92 percent of extant progress notes contained evidence of SUD treatment. However the majority of records with SUD clinic stop codes lacked both SUD diagnoses and relevant mental health CPT codes (53%), among which the specificity for documented SUD treatment was highest in records with a SUD diagnosis only (82.5%), lower for records with neither a diagnosis nor a mental health CPT code (55.3%), and lowest for records with only a mental health CPT code (21.9%). Beyond the variability of concordance between outpatient record types, substantial variability existed between facility within record types, both in concordance rates and rates of missing progress notes. These data should be considered by SUD researchers and quality managers in deciding how to operationalize a SUD encounter and the caveats regarding the metrics they choose.

Abbreviations: CPT = common procedural terminology, FY = fiscal year, HEDIS = Healthcare Effectiveness Data and Information Set, NPCD = National Patient Care Database, PTSD = posttraumatic stress disorder, SUD = substance use disorder, VA = Department of Veterans Affairs, VHA = Veterans Health Administration, VISN = Veterans Integrated Service Network, VistA = Veterans Health Information Systems and Technology Architecture.

ACKNOWLEDGMENTS

Author Contributions:

Study concept and design: A. H. S. Harris.

Acquisition of data: A. H. S. Harris, R. N. Reeder, L. Ellerbe, T. Bowe.

Analysis and interpretation of data: A. H. S. Harris, R. N. Reeder, L. Ellerbe, T. Bowe.

Drafting of manuscript: A. H. S. Harris, R. N. Reeder, L. Ellerbe.

Critical revision of manuscript for important intellectual content: A. H. S. Harris, R. N. Reeder, L. Ellerbe.

Statistical analysis: A. H. S. Harris, T. Bowe.

Obtained funding: A. H. S. Harris.

Study supervision: A. H. S. Harris.

Financial Disclosures: The authors have declared that no competing interests exist.

Funding/Support: This material was based on work supported by the Department of Veterans Affairs (VA) Office of Research and Development, Health Services Research and Development Service (grant IIR-07 092-1).

Additional Contributions: The views expressed in this article are those of the authors and do not necessarily represent the position of the VA.

Institutional Review: Approvals for all aspects of this study were obtained from the VA Palo Alto Research and Development Committee and the Stanford University Human Subjects Institutional Review Board.

REFERENCES

[1.] Joish VN, Malone DC, Wendel C, Mohler MJ. Profiling quality of diabetes care in a Veterans Affairs Healthcare System. Am J Med Qual. 2004;19(3):112-20. [PMID: 15212316] DOI:10.1177/106286060401900304

[2.] Wachter RM, Flanders SA, Fee C, Pronovost PJ. Public reporting of antibiotic timing in patients with pneumonia: Lessons from a flawed performance measure. Ann Intern Med. 2008;149(1):29-32. [PMID: 18591635]

[3.] Rosenheck R, Fontana A, Stolar M. Assessing quality of care: Administrative indicators and clinical outcomes in posttraumatic stress disorder. Med Care. 1999;37(2):180-88. [PMID: 10024122] DOI:10.1097/00005650-199902000-00008

[4.] Quan H, Parsons GA, Ghali WA. Validity of procedure codes in International Classification of Diseases, 9th revision, clinical modification administrative data. Med Care. 2004;42(8):801-9. [PMID: 15258482] DOI:10.1097/01.mlr.0000132391.59713.0d

[5.] Maynard C, Chapko MK. Data resources in the Department of Veterans Affairs. Diabetes Care. 2004;27(Suppl 2):B22-26. [PMID: 15113778] DOI:10.2337/diacare.27.suppl 2.B22

[6.] Kashner TM. Agreement between administrative files and written medical records: A case of the Department of Veterans Affairs. Med Care. 1998;36(9):1324-36. [PMID: 9749656] DOI:10.1097/00005650-199809000-00005

[7.] Humphreys K, Harris AH, Kivlahan DR. Performance monitoring of substance use disorder interventions in the Veterans Health Administration. Am J Drug Alcohol Abuse. 2009;35(3):123-27. [PMID: 19462294] DOI:10.1080/00952990802707042

[8.] VistAWeb Version 9.0 WEBV.1.16 User Manual. Washington (DC): Department of Veterans Affairs; 2009.

[9.] HEDIS 2006 Volume 2: Technical Specifications. Washington (DC): National Committee for Quality Assurance; 2006.

[10.] Stemler S. An overview of content analysis. Prac Assess Res Eval [Internet]. 2001;7(17). Available from: http://pareonline.net/getvn.asp?v=7&n=17

[11.] McTavish DG, Pirro EB. Contextual content analysis. Qual Quant. 1990;24(3):245-65. DOI:10.1007/BF00139259

[12.] Stange KC, Zyzanski SJ, Smith TF, Kelly R, Langa DM, Flocke SA, Jaen CR. How valid are medical records and patient questionnaires for physician profiling and health services research? A comparison with direct observation of patients visits. Med Care. 1998;36(6):851-67. [PMID: 9630127] DOI:10.1097/00005650-199806000-00009

[13.] Tisnado DM, Adams JL, Liu H, Damberg CL, Chen WP, Hu FA, Carlisle DM, Mangione CM, Kahn KL. What is the concordance between the medical record and patient self-report as data sources for ambulatory care? Med Care. 2006;44(2):132-40. [PMID: 16434912] DOI:10.1097/01.mlr.0000196952.15921.bf

[14.] Peabody JW, Luck J, Glassman P, Dresselhaus TR, Lee M. Comparison of vignettes, standardized patients, and chart abstraction: A prospective validation study of 3 methods for measuring quality. JAMA. 2000;283(13):1715-22. [PMID: 10755498] DOI:10.1001/jama.283.13.1715

[15.] Luck J, Peabody JW, Dresselhaus TR, Lee M, Glassman P. How well does chart abstraction measure quality? A prospective comparison of standardized patients with the medical record. Am J Med. 2000;108(8):642-49. [PMID: 10856412] DOI:10.1016/S0002-9343(00)00363-6

Submitted for publication July 29, 2009. Accepted in revised form February 22, 2010.

This article and any supplementary material should be cited as follows:

Harris AHS, Reeder RN, Ellerbe L, Bowe T. Are VHA administrative location codes valid indicators of specialty substance use disorder treatment? J Rehabil Res Dev. 2010;47(8):699-708.

DOI: 10.1682/JRRD.2009.07.0106

Alex H. S. Harris, PhD; * Rachelle N. Reeder, BA; Laura Ellerbe, MS; Thomas Bowe, PhD

Center for Health Care Evaluation, Department of Veterans Affairs Palo Alto Health Care System, Palo Alto, CA

* Address all correspondence to Alex H. S. Harris, PhD; VA Palo Alto Health Care System-Center for Health Care Evaluation (MC152), 795 Willow Road, Menlo Park, California 94025; 650-493-5000, ext 23423; fax: 650-617-2736. Email: Alexander.Harris2@va.gov
Table 1.
Treatment categories for records with documentation of SUD care.

Treatment Category                 Examples

Admission/Discharge Note from      Admitted to drug and alcohol
  Inpatient Stay with Documented   program for treatment of cocaine
  SUD Treatment                    dependence.

                                   Admitted for detoxification and
                                   scheduled to attend SUD-related
                                   groups or therapies.

                                   Admitted for medical condition
                                   (e.g., cellulitis) and
                                   consideration of treatment for
                                   alcohol relapse.

SUD Outpatient Care                Treatment in addiction setting
                                   (e.g., substance abuse treatment
                                   program) that is not gam-bling
                                   or smoking related.

                                   Social services (e.g., housing)
                                   provided in SUD specialty
                                   setting.

                                   Relapse prevention session.

Detoxification                     Detoxification is "chief
                                   complaint" or sole "reason for
                                   admission."

SUD Assessment                     Addiction Severity Index.

Outpatient Care Partially          Positive results on SUD-related
  Related to SUD                   screening (e.g., AUDIT-C, CAGE
                                   screening) and provider makes
                                   recommendation or takes further
                                   action.

                                   Dual diagnosis treatment (e.g.,
                                   seeking safety) for co-occurring
                                   PTSD and SUD.

                                   Treatment of SUD and nicotine
                                   dependence.

AUDIT-C = Alcohol Use Disorders Identification Test, PTSD =
posttraumatic stress disorder, SUD = substance use disorder.

Table 2.
Chart documentation of SUD treatment by record type.

                                    Total No.   No. (%) with
Record Type                          Records      Any Chart
                                    Selected    Documentation
                                                (VISN %Range)
Outpatient SUD Clinic

+ SUD Diagnosis,+ Procedure Code       700       601 (85.9)
(47% of all SUD Clinic Stop                        (67-96)
Records)

+ SUD Diagnosis, No Procedure          422       372 (88.2)
Code (40% of all SUD Clinic Stop                  (73-100)
Records)

No SUD Diagnosis, + Procedure          445       379 (85.2)
Code (8% of all SUD Clinic Stop                   (66-100)
Records)

No SUD Diagnosis, No Procedure         383       318 (83.0)
Code (5% of all SUD Clinic Stop                   (66-100)
Records)

Inpatient SUD Bed Section

+ SUD Diagnosis (99.9% of all SUD      700       699 (99.9)
Bed Section Records)                              (99-100)

No SUD Diagnosis (<0.1% of all         39        39 (100.0)
SUD Bed Section Records)

                                     No. with Chart     Percent of
Record Type                         Documentation of    Documented
                                     SUD Treatment        95% CI
                                                       (VISN %Range)
Outpatient SUD Clinic

+ SUD Diagnosis,+ Procedure Code          553              92.0
(47% of all SUD Clinic Stop                               (90-94)
Records)                                                 (82-100)

+ SUD Diagnosis, No Procedure             307              82.5
Code (40% of all SUD Clinic Stop                          (78-86)
Records)                                                 (57-100)

No SUD Diagnosis, + Procedure              83              21.9
Code (8% of all SUD Clinic Stop                           (19-26)
Records)                                                  (0-65)

No SUD Diagnosis, No Procedure            176              55.3
Code (5% of all SUD Clinic Stop                           (50-61)
Records)                                                  (23-81)

Inpatient SUD Bed Section

+ SUD Diagnosis (99.9% of all SUD         692              99.0
Bed Section Records)                                     (98-100)
                                                         (95-100)

No SUD Diagnosis (<0.1% of all             0                 0
SUD Bed Section Records)

                                    Percent of Total
Record Type                              95% CI
                                     (VISN %Range)

Outpatient SUD Clinic

+ SUD Diagnosis,+ Procedure Code          79.0
(47% of all SUD Clinic Stop             (76-82)
Records)                                (57-96)

+ SUD Diagnosis, No Procedure             72.7
Code (40% of all SUD Clinic Stop        (68-77)
Records)                                (53-95)

No SUD Diagnosis, + Procedure             18.6
Code (8% of all SUD Clinic Stop         (16-22)
Records)                                 (0-48)

No SUD Diagnosis, No Procedure            46.0
Code (5% of all SUD Clinic Stop         (41-51)
Records)                                (20-66)

Inpatient SUD Bed Section

+ SUD Diagnosis (99.9% of all SUD         98.9
Bed Section Records)                    (98-100)
                                        (95-100)

No SUD Diagnosis (<0.1% of all             0
SUD Bed Section Records)

CI = confidence interval, SUD = substance use disorder,
VISN = Veterans Integrated Service Network.

Table 3.
Treatment categories for outpatient records without documentation
of SUD care.

                               With +SUD       Without +SUD
                              Diagnosis,        Diagnosis,
Treatment Category          With +Procedure   With +Procedure
                             Code (n = 48)    Code (n = 504)

Non-SUD Mental Health             12                234

Smoking Cessation/Tobacco          7                 57
Counseling

Telephone Call or Letter           1                 50
(scheduling or
informational: not
telephone care)

Medicine (e.g., general,           8                 27
internal, dental)

Tx for Gambling                    0                 21

Tx Information (e.g.,              4                 17
schedule, structure)

Social Work or Services            2                 19
(including vocational)

Cancellation/No-Show               5                 14

Recreation or Exercise             1                 14
Therapy

--Screen, No Relapse               0                 15
Prevention

Medication Management              5                  9
(non-SUD)

Anger Management (non-             0                  6
SUD)

Domestic Violence (non-            0                  6
SUD setting)

+Screen, -Response                 2                  4

Homeless Case Management           1                  4

Nutrition                          0                  4

Immunization/Skin Test             0                  3

                            Total (n = 552)
Treatment Category               n (%)

Non-SUD Mental Health          246 (44.6)

Smoking Cessation/Tobacco       64 (11.6)
Counseling

Telephone Call or Letter        51 (9.2)
(scheduling or
informational: not
telephone care)

Medicine (e.g., general,        35 (6.3)
internal, dental)

Tx for Gambling                 21 (3.8)

Tx Information (e.g.,           21 (3.8)
schedule, structure)

Social Work or Services         21 (3.8)
(including vocational)

Cancellation/No-Show            19 (3.4)

Recreation or Exercise          15 (2.7)
Therapy

--Screen, No Relapse            15 (2.7)
Prevention

Medication Management           14 (2.5)
(non-SUD)

Anger Management (non-           6 (1.1)
SUD)

Domestic Violence (non-          6 (1.1)
SUD setting)

+Screen, -Response               6 (1.1)

Homeless Case Management         5 (0.7)

Nutrition                        4 (0.7)

Immunization/Skin Test           3 (0.5)

SUD = substance use disorder, Tx = treatment.

Figure.
Definition of Veterans Health Administration SUD specialty
locations, diagnoses, and procedures. Note: SUD specialty
code limited to those specified by SUD Continuity of Care
performance measure. Some older, rarely used codes are
omitted: note exclusion of 305.1--tobacco used disorder.
CPT = common procedural terminoloy, ICD-9-CM = International
Classification of Diseases-9th Revision-Clinical Modification,
PTSD = posttraumatic stress disorder, SUD = substance use
disorder.

Locations    Clinic Stops   SUD Individal Session (513
             (Outpatient)   SUD Home Visit (514)
                            SUD/PTSD (519)
                            Intensive SUD Treatment (547)
                            SUD Group Session (560)
                            SUD Telephone (545)

             Bed Sections   SUD Residential Rehabilitation (27)
             (Inpatient)    High-Intensity SUD Treatment Program (74)
                            SUD Domicilliary (86)

SUD                         291-292, 303-304, 305.0, 305.2-305.9,
Diagnoses                   535.3, 571.1
(ICD-9-CM)

Mental                      90801-90802, 90804-90815, 90826-90829,
Health                      90845-90847, 90849, 90853, 90857, 90862,
Related                     90870-90871, 90875-90876, 99201-99205,
Procedures                  99211-99215, 99217-99220, 99241-99245,
(CPT)                       99341-99345, 99347-9935, 99385-99387,
                            99395-99397, 99401-99404, 99420
Gale Copyright: Copyright 2010 Gale, Cengage Learning. All rights reserved.