Assessing Readmission to Substance Abuse Treatment as an Indicator of Outcome and Program Performance

This study is part of a large, ongoing evaluation project that has been described in detail elsewhere (8). The sample consists of 3,018 male substance abuse inpatients who were treated for 21 to 28 days between May 1992 and January 1994 at one of 15 Department of Veterans Affairs (VA) facilities and who underwent follow-up one year later. Given an intake sample of 3,612 patients, the follow-up rate was 84 percent.

Participants' mean±SD age at admission was 42.9±9.6 years, and they had a mean±SD of 12.7±1.8 years of education. The sample was primarily composed of African-American veterans (1,468, or 49 percent) and non-Hispanic Caucasian veterans (1,384, or 46 percent). At intake, only 24 percent were employed (N=717), and 18 percent were married (N=547). More than one-third of the participants (1,016, or 34 percent) had been involved in crime, as evidenced by one or more arrests in the previous 12 months. A total of 362 participants (12 percent) were classified as homeless at intake, defined as having spent most of the previous 12 months on the street or in shelters.

According to DSM-III-R diagnostic criteria, 41 percent (N=1,231) had a diagnosis of alcohol dependence or abuse only, 14 percent (N=417) had a diagnosis of drug dependence or abuse only, and 45 percent (N=1,356) were diagnosed as having both alcohol and drug dependence or abuse. In addition, 36 percent (N=1,096) had a comorbid axis I or axis II psychiatric diagnosis.

Patients who agreed to participate in the study completed an inventory at baseline and at follow-up that collected information on demographic characteristics and on four self-reported clinical outcomes, using the past three months as the reference window. The outcomes were any use of drugs or alcohol, coded dichotomously; negative consequences of substance use; depression or anxiety symptoms; and severe psychiatric symptoms. Negative consequences of substance abuse were measured as the sum of positive responses to 18 potential consequences such as family arguments and job problems; depression or anxiety symptoms were measured as the sum of the scores on 12 items from the depression and anxiety scales of the Brief Symptom Inventory (BSI) (9), each ranging from 0, not at all, to 4, extremely. Severe psychiatric symptoms were measured as the sum of the scores on ten items from the BSI paranoid ideation and psychoticism scales, each also ranging from 0 to 4.

Using national VA databases covering all facilities in the system, a dichotomous variable reflecting any readmission with a substance use diagnosis within one year of discharge was created. Because clinical outcome was assessed one year after discharge and thus may better correspond to later readmissions, four additional dichotomous variables were created. They reflected any readmission to inpatient substance abuse treatment within 90 days after discharge, 91 to 180 days after discharge, 181 to 270 days after discharge, and 271 to 360 days after discharge.

Of the 3,018 patients, 1,269 (42 percent) were readmitted at least once within a year after discharge; 788 patients (26 percent) were readmitted at least once within 90 days, 226 patients (8 percent) within 91 to 180 days, 175 patients (6 percent) within 181 to 270 days, and 121 patients (4 percent) within 271 to 360 days.

At one-year follow-up, patients had improved significantly on use of drugs and alcohol (98 percent of participants were using at baseline versus 60 percent at follow-up), negative consequences of substance use (mean±SD= 5.6±4.3 at baseline versus 3.1±3.9 at follow-up), depression and anxiety symptoms (20.1±11.3 at baseline versus 15.6±11.9 at follow-up), and severe psychiatric symptoms (14.2±8.6 at baseline versus 11.6±9 at follow-up). In addition, a higher proportion of patients were employed (38 percent versus 24 percent) and a lower proportion were involved in crime (22 percent versus 34 percent) at follow-up than at baseline. A detailed analysis of outcome data for this sample is presented elsewhere (8).

We considered an observed relationship between readmission and outcome measures significant if it reached Cohen's standard (10) for a "medium" effect size—a correlation of .3 or more, which would explain 9 percent or more of the shared variance. None of the point biserial and fourfold point correlations between readmission and outcome measures even approached this modest standard. Indeed, most were close to zero, and the largest explained only 2.5 percent of variance. We repeated these analyses for important subpopulations within the sample—those who at baseline were homeless, were unemployed, had a dual diagnosis, were involved in crime, were dependent on alcohol only, were dependent on drugs only, and were dependent on both alcohol and drugs. Again, readmission and outcome measures were essentially independent, and none of the 140 correlations we calculated even approached the standard of a medium effect size.

In a second set of analyses, the 15 inpatient programs were ranked according to their rates of readmission and adverse patient outcomes. The association between program rankings on readmission with rankings of abstinence, consequences of substance use, psychological distress, and psychiatric symptoms were all small and not statistically significant.

Readmission to inpatient substance abuse treatment is an inexpensively constructed index of utilization and cost, and it is certainly valuable as such. However, the results of this study indicate that it is essentially independent of one-year patient substance abuse and psychiatric outcomes and therefore should not be used as a substitute for direct assessment of clinical outcome and treatment quality.

The measurement error of readmission—produced by many readmitted patients experiencing good outcomes and many patients who were not readmitted experiencing bad outcomes—was found to be uniformly high. Lack of any observed readmission might indicate not that a patient has had a positive outcome but rather that the patient is incarcerated or is receiving care in a non-VA psychiatric facility. Similarly, rather than indexing poor clinical outcome, readmission might reflect the onset of cold weather, lack of affordable housing, or reductions in VA benefits.

These findings in a sample of substance abuse inpatients echo those of Lyons and colleagues (7) on readmitted psychiatric patients. Taken together, the two studies indicate that across patient populations readmission is not a valid indicator of clinical outcome or program performance.

This work was supported by the Department of Veterans Affairs mental health strategic health care group and health services research and development service. The authors thank John Finney, Ph.D., for his helpful comments on an earlier draft of this paper.

The authors are affiliated with the Program Evaluation and Resource Center at the Veterans Affairs Palo Alto Health Care System (VAPAHCS) and the department of psychiatry at Stanford University School of Medicine. Address correspondence to Dr. Humphreys at the Program Evaluation and Resource Center, VAPAHCS (152-MPD), 795 Willow Road, Menlo Park, California 94025 (e-mail, [email protected]).