Using Recidivism Rates as a Quality Indicator for Substance Abuse Treatment Programs

We analyzed electronic utilization records for all patients who received intensive inpatient substance abuse rehabilitation treatment or inpatient medical detoxification services provided by the Connecticut Department of Mental Health and Addiction Service at four state-operated psychiatric hospitals between July 1 and December 31, 1995. These services correspond to level III.7 and level IV, respectively, described by the American Society of Addiction Medicine (8). Index admissions were the first admission to the hospitals within the time period. If a patient had no further admissions within the same six months, his or her follow-up period was censored at January 1, 1996.

For multivariate analyses we used Poisson regression (5), a powerful, yet easy-to-use statistical technique that has three advantages for this application. First, like other forms of regression, Poisson regression allows for the statistical adjustment of many different factors. Second, it models the time to an event (person time), not the probability of occurrence. Measuring person time corrects for differing lengths of follow-up for each patient, thereby allowing the use of cross-sectional sampling. Third, person-time models can distinguish between two factors that both cause recidivism by indicating which one causes recidivism faster (and is therefore more harmful and results in more expense).

Allowing an individual to enter the study near the end of the study period is unlikely to bias person-time models of recidivism if the overall length of follow-up is long enough to catch most recidivist episodes (although it does bias probabilistic models such as logistic regression). Other studies indicate that most substance abuse readmissions occur within three to four months of discharge (6,7), suggesting that our six-month cross-section is the shortest period that is adequate for quality monitoring.

In our analyses, we modeled how the index admission and all the admission and patient characteristics known at the time of admission predicted the next episode of care in any of the state's four state hospitals. We defined the outcome of interest as the length of time to the first recidivist episode.

A recidivist episode was defined as any new episode of inpatient substance abuse treatment after the index episode. Specifically, if an initial detoxification was followed by rehabilitation treatment as part of the same treatment episode, the rehabilitation episode was counted as the index admission, and the detoxification admission and its associated variables were dropped from the analysis. Any subsequent admissions were coded as recidivist. This procedure of collapsing the detoxification and subsequent rehabilitation episode into one episode applied to 8.1 percent of the sample (N=204). In Table 1 it is referred to as a detoxification-rehabilitation episode.

This analysis also included data on inpatient, outpatient, and partial hospital substance abuse treatment provided by state facilities other than the four state-operated psychiatric hospitals between discharge from the index episode and either the censoring date or the date of admission for a recidivist episode. Information about such treatment is stored in a separate state database, which was matched with the study database by Social Security number. Social Security numbers were unavailable for less than 1 percent of the sample (N=21).

Using SAS 6.11 (9), Poisson regression modeling proceeded in a backward fashion. Variables were removed if they were not significant and had no confounding effect on other variables. Missing data were coded as distinct categories of independent variables to prevent bias caused by differences in information collection.

The sample consisted of 2,532 patients with a mean±SD age of 36.2± 9.8 years. The mean±SD length of stay was 13.6±21.3 days.

Table 1 presents the adjusted relative risk ratios from the regression model, along with the percentages of recidivism. The overall recidivism rate among the 2,532 patients was 18 percent (N=460). Eight percent of the sample (N=204) had two or more recidivist episodes; one patient was readmitted 14 times during the follow-up period. It should be noted that cross-sectional sampling introduces a negative bias to the percentages—that is, if each patient were followed for the full six months, the percentages would be higher. However, the adjusted relative risks are accurate and can be converted into absolute rates to monitor recidivism across reporting periods.

Sex and psychiatric diagnosis were not significant predictors or confounders of the recidivism-treatment facility relationship and were therefore dropped from the final model. Index discharge status and index episode were also not useful predictors of recidivism, but they did act as confounders. Missing data were the only categories that were significantly different from baseline for both the language and race-ethnicity variables, confirming and minimizing bias caused by faulty reporting. Patients most likely to have a recidivist episode were those who had received a physician's emergency commitment (risk ratio=1.71, p<.001), those who used both alcohol and opiates (risk ratio=1.67, p=.01), and those who were not married (risk ratio=1.62, p=.011). Patients with an alcohol and cocaine problem were 1.54 times less likely (p=.03) to have a recidivist episode than those who used alcohol only.

The most interesting finding was the difference in recidivism rates between the four hospitals. Based on Wald statistic contrasts (10), the recidivism rate at hospital D was significantly different from rates at the other three hospitals (χ² =6.22, df=1, p<.014). Clients discharged from that facility were 1.6 times more likely to have a recidivist episode than clients discharged from hospital A, which had the next highest recidivism rate. The rate at hospital B was different from the rate at hospital A (χ² =12.22, df=1, p=.001), but not from the rate at hospital C.

With statistical analyses like these, administrators of provider networks can address three aspects of quality. First, if recidivism rates are not found to differ among subgroups based on language, race, and sex, then recidivism is equally distributed among these subgroups. Such a finding suggests that these factors do not serve as barriers to readmission. Second, this type of analysis identifies service characteristics a network manager may wish to explore more deeply for improvement. In this study, patients who were involuntarily committed, who were not married, or who had alcohol and opiate problems at the index admission had the highest risk of recidivism. Such findings should be presented to clinical staff for further discussion of how to best address the needs of these groups.

Finally, a network manager should note whether the rate at any facility indicates an unusually high risk of recidivism—a finding that raises concerns about the quality of care in that institution. Multivariate modeling minimizes case mix differences and access to out-of-network care as explanations for an unusual rate, such as the one found for hospital D in our study. Two other possibilities remain for the higher rate at hospital D: that facility may have implemented its programs differently from its sister facilities, or some shortcoming in quality exists. The degree to which either of these hypotheses is true will have to be explored through other quality indicators and discussions with clinical and administrative staff.

Typical quality assurance analyses of recidivism present an unadjusted rate of readmission for a fixed time period. By capitalizing on modern information management systems and analytic software, we can enhance the explanatory power of recidivism measures by offering convenient sampling, case-mix-adjusted facility ranking, and presentation of risk ratios that account for both the speed and likelihood in recidivism. This specificity and depth give the network manager an important tool to help improve systems of care.

This research was sponsored in part by the Connecticut Department of Mental Health and Addiction Services.

When this work was done, Dr. Thakur was a doctoral candidate in the department of epidemiology and public health in the School of Medicine at Yale University in New Haven. He is now a National Institute of Mental Health postdoctoral fellow at the Cecil G. Sheps Center for Health Services Research of the University of North Carolina at Chapel Hill, 725 Airport Road, Chapel Hill, North Carolina 27599-7590 (e-mail, [email protected]). Dr. Hoff and Dr. Druss are assistant professors of psychiatry and public health at the Yale University School of Medicine. Dr. Hoff is also associate director of the practice pattern research unit at the Northeast Program Evaluation Center of the Veterans Affairs Connecticut Healthcare System in West Haven, Connecticut. Mr. Catalanotto is director of inpatient services for the Greater Bridgeport Mental Health Center in Bridgeport, Connecticut.