Impact of Agency Receipt of Incentives and Reminders on Engagement and Continuity of Care for Clients With Co-Occurring Disorders
Abstract
Objective:
This study examined whether having co-occurring substance use and mental disorders influenced treatment engagement or continuity of care and whether offering financial incentives, client-specific electronic reminders, or a combination to treatment agencies improved treatment engagement and continuity of care among clients with co-occurring disorders.
Methods:
The study used a randomized cluster design to assign agencies (N=196) providing publicly funded substance use disorder treatment in Washington State to a research arm: incentives only, reminders only, incentives and reminders, and a control condition. Data were analyzed for 76,044 outpatient, 32,797 residential, and 39,006 detoxification admissions from Washington’s treatment data system. Multilevel logistic regressions were conducted, with clients nested within agencies, to examine the effect of the interventions on treatment engagement and continuity of care.
Results:
Compared with clients with a substance use disorder only, clients with co-occurring disorders were less likely to engage in outpatient treatment or have continuity of care after discharge from residential treatment, but they were more likely to have continuity of care after discharge from detoxification. The interventions did not influence treatment engagement or continuity of care, except the reminders had a positive impact on continuity of care after residential treatment among clients with co-occurring disorders.
Conclusions:
In general, the interventions did not result in improved treatment engagement or continuity of care. The limited number of significant results supporting the influence of incentives and alerts on treatment engagement and continuity of care add to the mixed findings reported by previous research. Multiple interventions may be needed for performance improvement.
Many adults who report having a substance use disorder also report having a mental health problem in the past year (1). Individuals with co-occurring disorders often have more complex treatment needs (2), tend to have shorter stays in treatment and tend not to complete treatment (3,4), tend to be readmitted within a short period of time (5), and have less favorable outcomes compared with individuals who have a substance use disorder alone (6).
Research indicates that substance use disorder treatment can be improved through the use of performance measures to monitor treatment processes, such as treatment engagement and continuity of care. Meeting these measures is associated with improved outcomes for clients with substance use disorders (7–14). Treatment engagement occurs when ongoing treatment begins a short time after admission to outpatient treatment (15,16). Continuity of care occurs when clients from residential treatment or detoxification services receive at least one follow-up service shortly after discharge (15,17). The concept captured by these measures is that follow-up treatment should be timely and occur soon after the client enters outpatient treatment or leaves detoxification or residential care.
Lessons learned from research on substance use disorders may be used to improve treatment for those with co-occurring disorders. In fact, timely follow-up services (18,19) and continuity of care have been found to improve outcomes among individuals with co-occurring disorders, for example, by lowering mortality and reducing risk of early readmission (5,20).
Financial incentives and electronic reminders, interventions that have been used to improve treatment in general medicine (21–26) and substance use disorder treatment (27,28), may also be promising for individuals with co-occurring disorders. One of the few studies that used financial incentives to improve quality of mental health treatment showed that agency-level incentives increased the percentage of clients who received follow-up care shortly after assessment (29).
The main goal of this study was to examine the effects of treatment agency financial incentives and electronic reminders, referred to as “alerts” in our study, on treatment of clients with co-occurring substance use and mental disorders. We addressed three questions: whether having co-occurring disorders influences clients’ engagement in outpatient treatment or continuity of care after discharge from residential treatment or detoxification, whether agencies’ receipt of incentives or client-specific alerts influence treatment engagement or continuity of care, and whether combined receipt of client-specific alerts and incentives leads to additional improvement in performance beyond that of incentives only or electronic reminders only.
Methods
Agencies providing publicly funded outpatient or residential substance use disorder services to 25 or more clients between May 1, 2012, and April 30, 2013, in Washington State were assigned to four research arms (incentives only, alerts only, incentives and alerts, and a control group) by using a randomized cluster design. Because of their smaller number, detoxification agencies were randomly assigned to only three arms, omitting the incentives and alerts arm. The sampling frame comprised 151 outpatient, 12 detoxification, and 33 residential agencies. We stratified the randomization process to take into account the agencies’ baseline level of engagement in outpatient treatment or continuity of care after residential treatment or detoxification and number of admissions, an indicator of agency size. The study was approved by the Brandeis University and Washington State institutional review boards.
Data Sources
All data were from administrative data sources and included all publicly funded specialty treatment for substance use and mental disorders in the state, but they did not include primary care treatment. Client treatment data were obtained for individuals receiving state-funded substance use disorder treatment during the preintervention period (July 2012–September 2013) and the intervention period (October 2013–December 2015). Data from Washington’s Treatment Activity Report Generation Tool (TARGET) provided information on dates and types of services, information about clients’ demographic characteristics, employment, housing status, substance use history, and current substance use. Because the analytic file consists of client admissions, we use the term “clients” to refer to individual admissions. Mental health service data were merged with substance use disorder treatment data. The mental health file included data from Washington’s state hospital, community hospital, and outpatient databases, and, therefore, the assessments and service records contained in the file were from mental health facilities and not from substance use disorder treatment agencies.
Interventions
Incentives.
Treatment agencies in the incentives arm were eligible to receive financial incentives during each quarter on the basis of earned points. The incentives method was initially developed by researchers at the Brandeis/Harvard NIDA Center (30) and was implemented by the Centers for Medicare and Medicaid Services to reward hospitals for performance on a set of specified measures (31,32). Points could be earned based on quarterly performance rates in one of two ways.
Achievement and improvement points.
Agencies in the incentives arms could earn achievement and improvement points during each quarter on the basis of their quarterly performance relative to a minimum threshold and benchmark. The threshold and benchmark values were determined from the percentiles of agencies providing engagement or continuity of care during the baseline year. The achievement threshold was set at the 50th percentile, a reasonable lower bound for performance to be considered meritorious, and the benchmark level was set at the 90th percentile, a performance level that could reasonably be considered excellent. On the basis of baseline data, the achievement threshold and benchmark levels of continuity of care were set at 29% and 37%, respectively, for detoxification agencies and at 40% and 56%, respectively, for residential agencies. For outpatient agencies, the achievement threshold and benchmark levels of engagement were set at 76% and 90%, respectively.
Achievement points were awarded based on the extent to which an agency’s performance in a quarter exceeded the achievement threshold using equal intervals between the threshold and benchmark, with 1 point awarded for meeting the minimum achievement threshold and 10 points awarded for meeting or exceeding the benchmark level.
Treatment agencies could earn up to 10 improvement points each quarter for raising the rates of engagement or continuity of care above their own baseline rate, using equal intervals between their baseline value and the benchmark. An agency earned an improvement score of 0 if its rate was below its own baseline value (no improvement) and a score of 1 to 9 if its rate was in between (shows some improvement, but not yet up to benchmark).
Agencies were rewarded based on their achievement or improvement points, whichever were higher. For agencies with under 20 clients in a quarter, points were awarded in subsequent quarters when the count reached 20.
Alerts.
Alerts were delivered to treatment agencies in the form of an Excel workbook sent weekly through secure e-mail by Washington’s Behavioral Health Administration. The alerts showed the names of outpatient clients who were admitted in the past 45 days and clients who had been discharged from residential treatment and detoxification in the previous two weeks, the deadline for achieving engagement or continuity of care for each client, and the number of days until the deadline. The information was intended to “alert” agencies to clients who may be at risk of not meeting the engagement (outpatient treatment) or continuity of care (residential treatment and detoxification) criteria.
Variables
Dependent variables.
Treatment engagement was defined as receiving at least one service within 14 days of outpatient treatment admission (treatment initiation) and two or more services within 30 days of initiation (15,16).
Continuity of care was defined as receiving a treatment service within 14 days of being discharged from detoxification or residential treatment (15).
Co-occurring disorders.
A client was considered to have co-occurring substance use and mental disorders if he or she received one or more mental health services, reimbursed by state mental health funding, in the year prior to treatment admission.
Client-level covariates.
Outpatient and residential treatment analyses were controlled for client’s age, gender, race-ethnicity, education, marital status, employment status, past-month use of substances (alcohol, marijuana, cocaine, opioids, methamphetamines, and other drugs), age at first use of any substance, and homelessness status, based on clients’ self-report at admission. Other covariates included criminal justice involvement or referral, indicator variables for whether the client lived in a rural area, and indicator variables for whether clients had received substance use services in the year prior to admission. Detoxification models did not include education, marital status, employment status, and homelessness status, given that detoxification agencies collected less information.
Analyses
Analyses were conducted separately for each level of care by using individual, client-level data. For each level of care, two analyses were performed. First, a bivariate (unadjusted) analysis of the preintervention sample compared engagement and continuity of care rates for clients with co-occurring disorders and clients with a substance use disorder only. Second, a multilevel logistic regression (with clients nested within agencies) examined the effect of the interventions while controlling for client and agency characteristics. The multilevel regressions were performed separately by co-occurring status and used a difference-in-difference specification, which included indicators for the intervention arms, the intervention period, and their interaction. Significant interactions between the intervention arm and the intervention period indicated interventions with a significant effect on the likelihood of engagement or continuity of care. All analyses were conducted by using Stata, version 12.1.
Results
Table 1 shows characteristics of clients with substance use or co-occurring disorders by level of care. Among the 76,044 admitted outpatient clients, 29% received at least one mental health service in the year prior to admission, whereas 71% had a substance use disorder only. Of the 32,797 admitted residential clients, 36% had co-occurring disorders and 64% had a substance use disorder only. Of the 39,006 admitted detoxification clients, 34% had co-occurring disorders compared with 66% with a substance use disorder only.
| Characteristic | Outpatient (N=76,044) | Residential (N=32,797) | Detoxification (N=39,006) | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Co-occurring disorders (N=22,013) | Substance use disorder only (N=54,031) | Co-occurring disorders (N=11,691) | Substance use disorder only (N=21,106) | Co-occurring disorders (N=13,392) | Substance use disorder only (N=25,614) | |||||||
| N | % | N | % | N | % | N | % | N | % | N | % | |
| Female | 10,866 | 49.4 | 19,385 | 35.9 | 5,234 | 44.8 | 7,991 | 37.9 | 4,810 | 35.9 | 8,285 | 32.4 |
| Age | ||||||||||||
| 18–20 | 861 | 3.9 | 3,054 | 5.7 | 493 | 4.2 | 1,235 | 5.9 | 504 | 3.8 | 1,258 | 4.9 |
| 21–25 | 3,003 | 13.6 | 10,154 | 18.8 | 1,734 | 14.8 | 4,414 | 20.9 | 1,539 | 11.5 | 4,717 | 18.4 |
| 26–30 | 3,988 | 18.1 | 11,225 | 20.8 | 2,220 | 19.0 | 4,650 | 22.0 | 1,935 | 14.5 | 5,215 | 20.4 |
| 31–44 | 8,359 | 38.0 | 17,892 | 33.1 | 4,254 | 36.4 | 6,638 | 31.5 | 4,595 | 34.3 | 7,551 | 29.5 |
| ≥45 | 5,802 | 26.4 | 11,706 | 21.7 | 2,990 | 25.6 | 4,169 | 19.8 | 4,819 | 36.0 | 6,873 | 26.8 |
| Race-ethnicity | ||||||||||||
| White | 15,810 | 71.8 | 33,866 | 62.7 | 8,766 | 75.0 | 14,968 | 70.9 | 10,178 | 76.0 | 19,768 | 77.2 |
| Black | 2,296 | 10.4 | 3,720 | 6.9 | 1,067 | 9.1 | 1,206 | 5.7 | 991 | 7.4 | 1,435 | 5.6 |
| Latino | 1,770 | 8.0 | 6,188 | 11.5 | 758 | 6.5a | 1,448 | 6.9a | 1,163 | 8.7 | 1,787 | 7.0 |
| American Indian | 1,055 | 4.8 | 7,012 | 13.0 | 619 | 5.3 | 2,528 | 12.0 | 568 | 4.2 | 1,330 | 5.2 |
| Other | 1,082 | 4.9 | 3,245 | 6.0 | 481 | 4.1a | 956 | 4.5a | 492 | 3.7 | 1,294 | 5.1 |
| Educationb | ||||||||||||
| High school degree or GED | 12,205 | 55.4 | 30,471 | 56.4 | 6,643 | 56.8 | 12,342 | 58.5 | — | — | — | — |
| More than high school | 2,169 | 9.9 | 4,696 | 8.7 | 1,042 | 8.9 | 1,709 | 8.1 | — | — | — | — |
| Vocational | 1,367 | 6.2a | 3,342 | 6.2a | 610 | 5.2a | 1,020 | 4.8a | — | — | — | — |
| Marriedb | 4,078 | 18.5 | 13,583 | 25.1 | 1,755 | 15.0 | 4,087 | 19.4 | — | — | — | — |
| Homelessb | 4,310 | 19.6 | 6,599 | 12.2 | 4,144 | 35.5 | 6,500 | 30.8 | — | — | — | — |
| Employedb | 2,123 | 9.6 | 13,337 | 24.7 | 297 | 2.5 | 957 | 4.5 | — | — | — | — |
| Rural residence | 3,225 | 14.7 | 9,750 | 18.1 | 1,343 | 11.5 | 3,035 | 14.4 | 1,147 | 8.6 | 1,834 | 7.2 |
| Referred by or involved with criminal justice system | 10,797 | 49.1 | 33,572 | 62.1 | 4,531 | 38.8 | 7,679 | 36.4 | 793 | 5.9 | 507 | 2.0 |
| Substances used in past monthc | ||||||||||||
| Alcohol | ||||||||||||
| None | 8,987 | 40.8 | 22,303 | 41.3 | 4,821 | 41.2 | 10,322 | 48.9 | 5,939 | 44.4 | 14,948 | 58.4 |
| 1–3 times | 3,183 | 14.5 | 9,876 | 18.3 | 773 | 6.6a | 1,469 | 7.0a | 384 | 2.9 | 316 | 1.2 |
| 4–12 times | 2,544 | 11.6 | 7,691 | 14.2 | 841 | 7.2a | 1,577 | 7.5a | 500 | 3.7 | 471 | 1.8 |
| ≥13 times | 7,299 | 33.2 | 14,161 | 26.2 | 5,256 | 45.0 | 7,738 | 36.7 | 6,569 | 49.1 | 9,879 | 38.6 |
| Marijuana | ||||||||||||
| None | 12,296 | 55.9 | 31,792 | 58.8 | 6,758 | 57.8a | 11,997 | 56.8a | 10,750 | 80.3 | 21,115 | 82.4 |
| 1–3 times | 1,847 | 8.4a | 4,556 | 8.4a | 649 | 5.6a | 1,245 | 5.9a | 343 | 2.6a | 581 | 2.3a |
| 4–12 times | 1,563 | 7.1a | 3,942 | 7.3a | 715 | 6.1 | 1,498 | 7.1 | 343 | 2.6a | 602 | 2.4a |
| ≥13 times | 6,307 | 28.7 | 13,741 | 25.4 | 3,569 | 30.5a | 6,366 | 30.2a | 1,956 | 14.6 | 3,316 | 13.0 |
| Cocaine | 2,029 | 9.2 | 3,098 | 5.7 | 1,439 | 12.3 | 1,878 | 8.9 | 864 | 6.5 | 1,134 | 4.4 |
| Opiates | 5,637 | 25.6 | 15,345 | 28.4 | 4,564 | 39.0 | 10,597 | 50.2 | 4,232 | 31.6 | 14,793 | 57.8 |
| Methamphetamines | 8,353 | 38.0 | 16,347 | 30.3 | 5,831 | 49.9 | 10,209 | 48.4 | 4,562 | 34.1 | 7,050 | 27.5 |
| Other drug | 1,415 | 6.4 | 2,123 | 3.9 | 1,186 | 10.1 | 1,693 | 8.0 | 894 | 6.7 | 1,229 | 4.8 |
| Age of first used | ||||||||||||
| ≤10 | 3,437 | 15.6 | 5,789 | 10.7 | 1,890 | 16.2 | 2,639 | 12.5 | 1,635 | 12.2 | 2,160 | 8.4 |
| 11–14 | 8,602 | 39.1 | 19,149 | 35.4 | 4,817 | 41.2 | 8,373 | 39.7 | 4,473 | 33.4 | 6,668 | 26.0 |
| 15–17 | 5,860 | 26.6 | 17,063 | 31.6 | 2,948 | 25.2 | 5,884 | 27.9 | 3,037 | 22.7 | 6,066 | 23.7 |
| 18–20 | 2,155 | 9.8 | 6,570 | 12.2 | 1,031 | 8.8 | 2,050 | 9.7 | 1,644 | 12.3 | 4,022 | 15.7 |
| ≥21 | 1,959 | 8.9 | 5,460 | 10.1 | 1,005 | 8.6 | 2,160 | 10.2 | 2,603 | 19.4 | 6,698 | 26.2 |
| Prior treatment | ||||||||||||
| Outpatient | 6,746 | 30.7 | 11,800 | 21.8 | 4,772 | 40.8 | 7,024 | 33.3 | 4,028 | 30.1 | 4,310 | 16.8 |
| Residential | 6,270 | 28.5 | 10,838 | 20.1 | 3,408 | 29.2 | 4,729 | 22.4 | 3,552 | 26.5 | 3,602 | 14.1 |
| Detoxification | 3,662 | 16.6 | 5,518 | 10.2 | 4,796 | 41.0 | 8,200 | 38.9 | 7,367 | 55.0 | 9,981 | 39.0 |
a
All pairwise comparisons between clients with co-occurring and substance use disorders only in the same level of care were significant (p<.05), except comparisons indicated.
b
Information not collected in detoxification admissions
c
Substance was listed as a primary, secondary, or tertiary drug of abuse, and frequency of use was 1 or more times in the past month or during the month with highest use in the last year; frequency was from whichever month was the highest.
d
Earliest age of first use of any of the substances reported as the primary, secondary, or tertiary substance of abuse
Table 1 also indicates differences between clients with co-occurring disorders or a substance use disorder only. For example, compared with clients with a substance use disorder only, a higher proportion of clients with co-occurring disorders were women and were older (ages 31–44 and ≥45). This was true in all levels of care. Pairwise comparisons within each level of care showed that clients with co-occurring disorders differed significantly from clients with a substance use disorder on nearly every demographic variable. Only a few of the pairwise comparisons of clients in the same level of care were not significantly different (noted in Table 1). However, the preponderance of significant differences was due to the large sample sizes for each level of care.
Outcome Differences by Co-Occurring Status
We compared unadjusted outcomes for preintervention samples of clients with co-occurring disorders and clients with a substance use disorder only. Among outpatient clients, those with co-occurring disorders demonstrated a significantly lower rate of engagement (N=5,679, 27.3%) than did those who had a substance use disorder only (N=15,126, 72.7%) (p<.001). Among residential clients, the rate of continuity of care was significantly lower among clients with co-occurring disorders (N=1,749, 30.7%) than among clients with a substance use disorder only (N=3,948, 69.3%) (p<.001). Among detoxification clients, the rate of continuity of care was significantly lower among clients with co-occurring disorders (N=1,527, 33.6%) than among clients with a substance use disorder only (N=3,021, 66.4%) (p<.001).
For the same preintervention samples, we ran multilevel logistic regressions to estimate the effects of co-occurring mental disorders by level of care. The analyses were adjusted for client and agency characteristics (data not shown). These models provided similar estimates of the effects of co-occurring disorders on our outcomes, except that detoxification clients with co-occurring disorders tended to have higher rates of continuity of care than those who had a substance use disorder only.
Impact of Interventions on Clients With Co-Occurring Disorders
To examine whether the interventions had different effects on clients with co-occurring disorders, regressions were conducted separately for clients with co-occurring disorders and those who had a substance use disorder only. Table 2 shows the impact of interventions on each group by level of care. Incentives or alerts had no significant effect on engagement in outpatient treatment or on continuity of care after discharge from detoxification for any group of clients, regardless of whether they had a co-occurring disorder.
| Intervention | Continuity of care | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Treatment engagement (outpatient) (N=76,044) | Residential (N=32,797) | Detoxification (N=39,006)b | ||||||||||
| Co-occurring disorders (N=22,013) | Substance use disorder only (N=54,031) | Co-occurring disorders (N=11,691) | Substance use disorder only (N=21,106) | Co-occurring disorders (N=13,392) | Substance use disorder only (N=25,614) | |||||||
| Estimate | 95% CI | Estimate | 95% CI | Estimate | 95% CI | Estimate | 95% CI | Estimate | 95% CI | Estimate | 95% CI | |
| Intervention arm (reference: control arm) | ||||||||||||
| Alerts only | –.10 | –.51 to .30 | –.11 | –.49 to .28 | .01 | –.35 to .37 | .21 | –.12 to .54 | .09 | –.48 to .67 | –.02 | –.50 to .45 |
| Incentives only | .23 | –.18 to .65 | .06 | –.30 to .42 | .30 | –.06 to .65 | .41 | –.04 to .86 | –.18 | –.91 to .56 | –.50 | –1.43 to .42 |
| Incentives and alerts | .13 | –.28 to .54 | .02 | –.38 to .43 | .16 | –.16 to .48 | .20 | –.11 to .51 | — | — | — | — |
| Intervention period (reference: preintervention) | .14 | –.12 to .40 | .06 | –.07 to .19 | –.19* | –.35 to –.02 | –.15 | –.37 to .06 | .20 | .00 to .40 | .05 | –.07 to .18 |
| Intervention arm and intervention period interaction | ||||||||||||
| Alerts only × intervention period | .02 | –.31 to .34 | –.05 | –.25 to .16 | .32* | .05 to .60 | .01 | –.25 to .26 | –.18 | –.43 to .08 | .01 | –.19 to .21 |
| Incentives only × intervention period | –.17 | –.50 to .16 | –.07 | –.27 to .12 | .14 | –.12 to .40 | .04 | –.26 to .34 | –.33 | –.68 to .02 | .14 | –.25 to .52 |
| Incentives and alerts × intervention period | –.09 | –.43 to .24 | –.01 | –.21 to .18 | –.11 | –.35 to .13 | –.03 | –.33 to .26 | — | — | — | — |
a
Results are from difference-in-difference regression models that controlled for client and facility covariates.
b
Because of the small number of detoxification agencies, they were randomly assigned to only three research arms (excluding the incentives and alerts research arm).
*
p<.05
Analysis of continuity after residential care, however, showed a significant positive interaction between the alerts-only research arm and the intervention time period (estimate=.32; 95% confidence interval [CI]=.05 to .60, p<.05). This result implied that the difference in likelihood between receiving continuity of care during the intervention compared with before the intervention was significantly larger among the alerts-only group than among the comparison group. Because the regression model also showed a significant negative main effect for the intervention period (estimate=–.19; CI=–.35 to –.02, p<.05), this significant positive interaction indicated that the alerts counteracted what was otherwise a decreasing likelihood of continuity of care over time. That is to say, providing residential agencies with alerts appeared to counteract an overall decline in continuity of care during the intervention period. Specifically, the alerts informed residential agencies regarding which clients were approaching 14 days since discharge without having received a follow-up treatment service and, therefore, were at risk of not meeting the continuity-of-care measure. However, the alerts were helpful in achieving continuity of care only for residential clients with co-occurring disorders. No similar impact was found for residential clients who had a substance use disorder only, even though they experienced an overall decline in continuity of care during the intervention period that was almost as large as that of the clients with co-occurring disorders.
Discussion
Clients with co-occurring substance use and mental disorders were found to be less likely than clients with a substance use disorder only to engage in outpatient treatment or to experience continuity of care after discharge from residential treatment. Detoxification clients who had co-occurring disorders were more likely than those with a substance use disorder only to have continuity of care. Individuals with mental health problems tend to have multiple detoxifications and utilize more services (33,34), which results in higher rates of continuity of care.
Our interventions generally had no impact on outpatient treatment engagement or on continuity of care after detoxification and residential discharge. However, an impact was found for the use of alerts in one group of clients—those with co-occurring disorders who were treated at residential agencies in the alerts-only arm. Because residential treatment is a more intense level of care (35), it tends to admit fewer clients than other levels of care, and staff may be better able to attend to client information, such as the alerts.
This finding somewhat duplicates previous research that found that use of electronic reminders for a recommended brief intervention was more likely to occur with clients who had more severe unhealthy alcohol use than with those who had mild or moderate alcohol problems or clients with mental health conditions (36,37). It appears that treatment staff focus more on clients with more complex needs to ensure that they receive needed follow-up services.
In addition, previous research found that certain types of practitioners may be more inclined than others to use the information from alerts. For example, nurse practitioners tend to use electronic reminders more than other staff (36). The tendency of residential treatment agencies to have more nurses on staff compared with outpatient agencies (38) may explain why implementation of alerts demonstrated an interaction effect for the former level of care.
Research has been somewhat mixed regarding the effectiveness of alerts and incentives in the areas of general medical health and behavioral health (23,27,28). Our results do little to clarify the influence of these interventions, given that our regression models found only limited situations in which alerts or incentives had a significant impact on treatment engagement and continuity of care. These findings suggest that generally no single intervention alone, such as alerts or incentives, is sufficient to bring about desired improvements. Additional interventions may often be needed to improve agency performance across the board. It has been found that information delivered in electronic reminders is used more readily by clinicians when the reminders are aligned with performance measures supported by the leadership (36). In Washington, treatment retention, not continuity of care, is the measure supported by state leadership. Although treatment agencies agree that continuity of care is important, the fact that Washington focuses on a retention measure may well explain why our study results were so modest.
There were several limitations of the study. First, the data were from one state, and, therefore, the ability to generalize to other states is limited. Second, although administrative data are a rich resource that include information for all clients who receive treatment in the state, the TARGET database captures only publicly treated clients. To the extent that agencies might treat state-funded clients differently than other clients, the results are again limited with regard to generalizability. Third, our definition of co-occurring substance use and mental disorders is based on clients’ service claims in the prior year. We used receipt of mental health services in the year prior to admission as a marker for having a mental health problem because the diagnosis data had a very high percentage of missing values. Because of the diagnosis data quality, we were not able to use that data to distinguish the various types and level of severity of mental health problems among clients who had a co-occurring disorder. Although everyone who received a mental health service in the prior year may not have had a mental health problem during the study period, we found that the great majority of clients who received a mental health service in the prior year continued to receive additional mental health services during the study period. Data completeness, however, may have resulted in an underestimation of the number of clients who had co-occurring disorders.
As a fourth limitation, our study was conducted while Washington State was transitioning to a system of behavioral health organizations. As a result, the agencies’ efforts were focused to a large extent on adapting to the new system, and they had reduced time and resources to focus on improving engagement and continuity of care. Finally, the use of electronic reminders could result in performance improvement when the desired outcome, such as conducting screening, is under the control of the agency. However, reminders may be less helpful when the outcome is not under the control of the agency, such as getting clients into follow-up care when the system is at capacity.
Conclusions
The implementation of electronic reminders, or alerts, was associated with a greater likelihood of continuity of care after residential treatment for clients with co-occurring disorders but not for those with a substance use disorder only. Financial incentives had no influence on continuity of care after residential treatment. Neither incentives nor alerts had any impact on engagement among outpatients or continuity care among detoxification clients. Limitations notwithstanding, to our knowledge, there has not been a randomized controlled study that examined the influence of incentives and alerts, together and separately, on treatment agency performance measures among clients with co-occurring disorders. The research on the impact of incentives on performance has been mixed and suggests that multiple interventions may be needed to bring about improvements and that the impact of these interventions may vary with different populations.
Acknowledgments
The authors appreciate the contributions of Jason Bean-Mortinson, Can Du, Alice Huber, Eric Larson, Katie Weaver-Randall, and Fritz Wrede.
Footnote
The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH or of Washington State.
References
1.
Key Substance Use and Mental Health Indicators in the United States: Results From the 2015 National Survey on Drug Use and Health. DHHS pub no SMA 16-4984, NSDUH series H-51. Substance Abuse and Mental Health Services Administration, Center for Behavioral Health Statistics and Quality, 2016. www.samhsa.gov/data/sites/default/files/NSDUH-FFR1-2015/NSDUH-FFR1-2015/NSDUH-FFR1-2015.pdf
2.
Rush B, Koegl CJ: Prevalence and profile of people with co-occurring mental and substance use disorders within a comprehensive mental health system. Canadian Journal of Psychiatry 53:810–821, 2008
3.
González-Saiz F, Vergara-Moragues E, Verdejo-García A, et al: Impact of psychiatric comorbidity on the in-treatment outcomes of cocaine-dependent patients in therapeutic communities. Substance Abuse 35:133–140, 2014
4.
Choi S, Adams SM, MacMaster SA, et al: Predictors of residential treatment retention among individuals with co-occurring substance abuse and mental health disorders. Journal of Psychoactive Drugs 45:122–131, 2013
5.
Ilgen MA, Hu KU, Moos RH, et al: Continuing care after inpatient psychiatric treatment for patients with psychiatric and substance use disorders. Psychiatric Services 59:982–988, 2008
6.
Park TW, Cheng DM, Samet JH, et al: Chronic care management for substance dependence in primary care among patients with co-occurring disorders. Psychiatric Services 66:72–79, 2015
7.
Garner BR, Godley MD, Funk RR, et al: The Washington Circle continuity of care performance measure: predictive validity with adolescents discharged from residential treatment. Journal of Substance Abuse Treatment 38:3–11, 2010
8.
Garnick DW, Horgan CM, Acevedo A, et al: Criminal justice outcomes after engagement in outpatient substance abuse treatment. Journal of Substance Abuse Treatment 46:295–305, 2014
9.
Garnick DW, Horgan CM, Lee MT, et al: Are Washington Circle performance measures associated with decreased criminal activity following treatment? Journal of Substance Abuse Treatment 33:341–352, 2007
10.
Lee MT, Horgan CM, Garnick DW, et al: A performance measure for continuity of care after detoxification: relationship with outcomes. Journal of Substance Abuse Treatment 47:130–139, 2014
11.
Mark TL, Vandivort-Warren R, Montejano LB: Factors affecting detoxification readmission: analysis of public sector data from three states. Journal of Substance Abuse Treatment 31:439–445, 2006
12.
Dunigan R, Acevedo A, Campbell K, et al: Engagement in outpatient substance abuse treatment and employment outcomes. Journal of Behavioral Health Services and Research 41:20–36, 2014
13.
Garnick DW, Lee MT, O’Brien PL, et al: The Washington Circle engagement performance measures’ association with adolescent treatment outcomes. Drug and Alcohol Dependence 124:250–258, 2012
14.
Harris AH, Gupta S, Bowe T, et al: Predictive validity of two process-of-care quality measures for residential substance use disorder treatment. Addiction Science and Clinical Practice 10:22, 2015
15.
Garnick DW, Lee MT, Horgan CM, et al: Adapting Washington Circle performance measures for public sector substance abuse treatment systems. Journal of Substance Abuse Treatment 36:265–277, 2009
16.
National Committee for Quality Assurance: HEDIS 2017. vol. 2. Summary Table of Measures, Product Lines, and Changes, 2017. www.ncqa.org/portals/0/hedisqm/hedis2017/hedis%202017%20volume%202%20list%20of%20measures.pdf?ver=2016-06-27-135433-350
17.
The ASAM Performance Measures for the Addiction Medicine Specialist. Chevy Chase, MD, American Society of Addiction Medicine, 2014. http://www.asam.org/docs/default-source/advocacy/performance-measures-for-the-addiction-specialist-physician.pdf?sfvrsn=0#search=%22detoxification%20follow%20up%22
18.
Wierdsma A, Mulder C, de Vries S, et al: Reconstructing continuity of care in mental health services: a multilevel conceptual framework. Journal of Health Services Research and Policy 14:52–57, 2009
19.
Fortney J, Sullivan G, Williams K, et al: Measuring continuity of care for clients of public mental health systems. Health Services Research 38:1157–1175, 2003
20.
Watkins KE, Paddock SM, Hudson TJ, et al: Association between quality measures and mortality in individuals with co-occurring mental health and substance use disorders. Journal of Substance Abuse Treatment 69:1–8, 2016
21.
Feldstein A, Elmer PJ, Smith DH, et al: Electronic medical record reminder improves osteoporosis management after a fracture: a randomized, controlled trial. Journal of the American Geriatrics Society 54:450–457, 2006
22.
Garg AX, Adhikari NK, McDonald H, et al: Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA 293:1223–1238, 2005
23.
Ivers N, Jamtvedt G, Flottorp S, et al: Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews 6:CD000259, 2012
24.
Dexheimer JW, Talbot TR, Sanders DL, et al: Prompting clinicians about preventive care measures: a systematic review of randomized controlled trials. Journal of the American Medical Informatics Association 15:311–320, 2008
25.
Bryan C, Boren SA: The use and effectiveness of electronic clinical decision support tools in the ambulatory/primary care setting: a systematic review of the literature. Informatics in Primary Care 16:79–91, 2008
26.
Schedlbauer A, Prasad V, Mulvaney C, et al: What evidence supports the use of computerized alerts and prompts to improve clinicians’ prescribing behavior? Journal of the American Medical Informatics Association 16:531–538, 2009
27.
Bremer RW, Scholle SH, Keyser D, et al: Pay for performance in behavioral health. Psychiatric Services 59:1419–1429, 2008
28.
Hull P, Ritter A: The Potential Role of Pay-for-Performance in Alcohol and Other Drug Treatment Funding: A Literature Review. Sydney, University of New South Wales, National Drug and Alcohol Research Centre, Drug Policy Modelling Program, 2014
29.
Unützer J, Chan YF, Hafer E, et al: Quality improvement with pay-for-performance incentives in integrated behavioral health care. American Journal of Public Health 102:e41–e45, 2012
30.
Tompkins C, Perloff J, Horgan C, et al: A randomized trial of financial incentives for substance abuse treatment providers in Connecticut. Presented at the Addiction Health Services Research Conference, San Francisco, October 28–30, 2009
31.
Report to Congress: Plan to Implement a Medicare Hospital Value-Based Purchasing Program. Baltimore, US Department of Health and Human Services, Centers for Medicare and Medicaid Services, 2007
32.
Pay-for-performance. Health Affairs, October 11, 2012 (doi 10.1377/hpb20121011.90233)
33.
Pasic J, Russo J, Roy-Byrne P: High utilizers of psychiatric emergency services. Psychiatric Services 56:678–684, 2005
34.
Amodeo M, Lundgren L, Chassler D, et al: High-frequency users of detoxification: who are they? Substance Use and Misuse 43:839–849, 2008
35.
McNeese-Smith D, Nyamathi A, Longshore D, et al: Processes and outcomes of substance abuse treatment between two programs for clients insured under managed care. American Journal of Drug and Alcohol Abuse 33:439–446, 2007
36.
Williams EC, Achtmeyer CE, Kivlahan DR, et al: Evaluation of an electronic clinical reminder to facilitate brief alcohol-counseling interventions in primary care. Journal of Studies on Alcohol and Drugs 71:720–725, 2010
37.
Williams EC, Lapham G, Achtmeyer CE, et al: Use of an electronic clinical reminder for brief alcohol counseling is associated with resolution of unhealthy alcohol use at follow-up screening. Journal of General Internal Medicine 25(suppl 1):11–17, 2010
38.
Knudsen HK, Abraham AJ: The employment of nurses in publicly funded substance abuse treatment programs. Journal of Addictions Nursing 23:174–180, 2012
Information & Authors
Information
Published In
Cover: Red Umbrella, by Milton Avery, 1945. Oil on canvas. Gift of Annalee Newman, Princeton Art Museum. Photo credit: Bruce M. White, Princeton University Art Museum/Art Resource. © The Milton Avery Trust/Artists Rights Society, New York City.
History
Received: 23 October 2017
Revision received: 2 February 2018
Accepted: 5 March 2018
Published online: 26 April 2018
Published in print: July 01, 2018
Keywords
Authors
Competing Interests
The authors report no financial relationships with commercial interests.
Funding Information
National Institute on Drug Abuse10.13039/100000026: P30 DA035772, R01DA033468
This research was supported by the National Institute on Drug Abuse (NIDA), National Institutes of Health (NIH), under award R01 DA033468 and is a component project of the NIDA-supported Brandeis/Harvard Center to Improve System Performance of Substance Use Disorder Treatment (P30 DA035772).Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).