The Effectiveness of Recovery-Oriented ACT in Reducing Hospital Use: Do Effects Vary Over Time?
Abstract
Objective
A previous study of recovery-oriented assertive community treatment (PACT) found large differences over three years in use of state psychiatric hospitals between PACT participants and consumers in a matched control group, especially for PACT participants with significant previous psychiatric hospitalization. This study extended these findings by examining the timing of PACT effects.
Methods
Generalized estimating equation models of monthly cost data for state, local, and crisis hospital use estimated the time-varying effects of participation in one of ten PACT teams in Washington State. Data from PACT participants (N=450) and propensity score–matched consumers (N=450) were included. Additional analyses determined whether effects differed by prior state hospital use.
Results
Differences in costs between PACT and control participants were largest immediately after PACT enrollment and tapered off. During the first quarter after enrollment, monthly per-person costs for state hospital use were $3,458 lower for PACT enrollees than for control participants. A composite measure of psychiatric hospital costs (state and local hospitals and local crisis stabilization units) declined by $3,539 monthly during the first quarter after PACT enrollment (p<.01). Differences were noted up to 27 months after enrollment, when the difference in the composite costs measure became insignificant compared with the prior quarter (months 25–27) (p>.05). Differences were larger for PACT enrollees with greater baseline state hospital use.
Conclusions
The time-varying estimates may have implications for the length and intensity of ACT enrollment. However, the optimum time for receipt of ACT services needs to be considered in the context of outcomes other than hospitalization alone.
From its inception in the early 1970s, assertive community treatment (ACT) has been viewed as a time-unlimited treatment program for persons with severe mental illness (1,2). This “ACT-for-life” rationale was based on several considerations, including level of disability, the episodic nature of the illness, and the intensity of the services required to help people to live in the community outside institutional settings. Empirical support came from the original clinical trial of ACT (3), in which individuals discharged after 18 months of services because of the end of grant funding returned to their previous levels of hospital use in the following 14 months (4).
As the model was disseminated across the country, however, state mental health authorities and program managers began to question the unlimited-duration assumption because of the per-capita costs of ACT and the capacity constraints associated with its deployment. Under an “ACT-for-life” mandate, once an ACT team recruited its full caseload of 60–100 consumers, the only way it was able to accommodate any new referrals was through death of enrolled consumers or their move to another area. This was in marked contrast to most other mental health services that dealt with a regular flow of admissions and discharges. Whereas a psychiatric inpatient unit or clinic, for example, might treat thousands of individuals over the course of a year, an ACT team with an unlimited-duration policy would be able to care only for its initial caseload of 60–100 enrollees.
Adherence to the unlimited-duration precept began to be questioned when both clinicians and advocates recognized that some consumers who were enrolled in ACT achieved improved functioning over time and no longer required the intensity of services that ACT routinely provided. For these consumers, indefinite retention may not be consistent with principles of recovery (5). These concerns about cost and personal growth have stimulated a number of studies about the opportunities and outcomes associated with ACT graduation, transitioning, or step-down to less intensive services (6–8). In fact, states such as New Mexico have provided two “tracks” for ACT services, at different levels of intensity and funding (9).
The related issue of how much exposure to ACT is required to achieve positive outcomes has received less attention. One uncontrolled study of ACT found that the biggest drop in days of psychiatric hospitalization occurred after the first year (7). A report from the ACCESS study of homeless individuals with mental illness, in which participants were enrolled in ACT services for only one year and then transitioned to other community services, found that selected participants were able to transition without subsequent loss of gains in mental health status, substance abuse, housing, or employment (8). But there have been few controlled efforts to model the timing and trajectory of consumer outcomes after ACT enrollment. Interest has recently been increasing in “critical time interventions” after psychiatric hospitalization (10–12) and prison release (13), but the critical time idea is largely foreign to ACT research because of the intensity of ACT services and assumptions about indefinite enrollment.
Our study of Washington State’s PACT, a recovery-oriented ACT initiative, found large differences in the use of state psychiatric hospitals over a 36-month period in contrast to matched controls, especially among participants with high levels of prior psychiatric hospital use (14). This study expanded on those findings to determine the timing of PACT effects over a three-year period.
Methods
Sample
This study examined data for individuals who participated in one of ten PACT teams established by Washington’s State Mental Health Authority (15). The first clients were enrolled in July 2007. Details about the PACT teams, state psychiatric hospitals, and the data sources in this study are described in a companion article (14) and are only briefly summarized below.
Of the 636 PACT participants with a valid match in the administrative data assembled for this study, we selected the 450 participants with a history of state psychiatric hospital utilization and matched them on the basis of propensity scores with 450 participants who also had prior state hospital use but did not receive PACT (control group).
Data and variables
Statewide, linked administrative data were obtained from the Research and Data Analysis Division, Washington State Department of Social and Health Services (DSHS). The database included DSHS client service contacts since 2000, measures derived from Medicaid claims data, demographic characteristics, diagnostic information, and service costs (16). Comprehensive data on outpatient costs were not available for our study.
All data were collapsed to the person-month level, with each observation reflecting use of services during that calendar month. A balanced panel of 111 monthly observations (January 2001 to March 2010) was included for each of the 900 study participants, with up to 33 monthly observations in the postenrollment period (post period), depending on the timing of PACT enrollment. That is, the first PACT enrollees began receiving PACT services in July 2007 and had 33 months of post period data and 6.5 years of pre-PACT data, whereas subsequent PACT enrollees contributed more pre-PACT observations and fewer observations in the post period.
Study design and analysis methods
We used an observational study design to compare PACT enrollees and propensity score–matched consumers in the control group. Consumers in the control group were randomly assigned a start date for their post period, based on the distribution of actual start dates from the PACT sample (17). The propensity score model, balance, and matching process are described in a companion article (14). In short, we used a set of covariates in the propensity models describing demographic characteristics, diagnoses, and patterns of use of a variety of services offered in Washington State up to four years before PACT enrollment. The propensity (logit) model passed a number of specification and balancing tests (18). Control group participants were selected by using nearest-neighbor techniques. Because the results from the previous study were quite similar regardless of whether nearest neighbors in the control group were required to have an estimated propensity score within a fixed radius or caliper, in this article we report results only for the full PACT sample (N=450) and the propensity score–matched consumers in the control group (N=450).
Outcome measures included state hospital costs alone and a composite measure of psychiatric stay costs obtained by multiplying days spent in a local psychiatric hospital and days spent in local crisis stabilization units by costs approximating per diem reimbursement rates ($1,166 and $600, respectively [14]) and adding these monthly costs to state psychiatric hospital costs. Cost measures were estimated with generalized estimating equations with an AR1 correlation structure. We ran separate models of both cost measures, which allowed for a separate effect of PACT for participants with a substantial history of state psychiatric hospitalization (defined as ≥96 days in the previous two years [19]).
Our primary interest was in measuring whether the effect of PACT varied over time since enrollment. We therefore estimated a time-varying effect of PACT by creating quarterly indicators of time since PACT enrollment—or the start of the post period for consumers in the control group—and interacted these quarterly time periods with the PACT indicator variable. From these models, we generated the average marginal effect of PACT enrollment for each time period. [The full set of model estimates is available from the authors.]
We also examined whether the PACT teams exhibited learning. Learning would have occurred if PACT participants enrolled later in the study period had better outcomes (lower costs) than earlier PACT enrollees, with the analysis controlling for pre-PACT characteristics. Alternatively, enrollees of PACT teams may have exhibited proportionately worse outcomes over time if, as time went on, the teams selected participants less suitable for PACT. We therefore included a variable reflecting time since the first PACT participants were enrolled and interacted this variable with the PACT indicator; the direction of the marginal effect of time for PACT teams allowed us to determine which of the two effects (learning or differential selection) had a larger effect, if any. In all analyses, we controlled for an array of pre-ACT inpatient and outpatient service utilization measures, demographic characteristics, diagnoses, and region of Washington State (east or west). The research was conducted with the approval of Institutional Review Boards at University of North Carolina and the Washington State Department of Social and Health Services.
Results
As shown in Table 1, control group participants were similar to PACT enrollees on all observable baseline characteristics and service utilization indicators. Both groups had a mean age of 40, and about 60% were male. In the overall sample (PACT and control group participants), approximately 77% were white, 13% were African American, and just over 2% were Latino. The vast majority (over 97%) had a diagnosis of schizophrenia. Almost three-fourths also had a diagnosis of an affective disorder, and just over half had a substance use diagnosis, indicating a high level of comorbidity in this population.
| Variable | PACT participants (N=450) | Control group (N=450) | ||
|---|---|---|---|---|
| N | % | N | % | |
| Age (M±SD) | 40±12 | 41±12 | ||
| Female | 172 | 38 | 178 | 40 |
| Race-ethnicity | ||||
| White | 356 | 79 | 347 | 77 |
| African American | 59 | 13 | 61 | 14 |
| Latino | 11 | 2 | 18 | 4 |
| Diagnosis | ||||
| Schizophrenia | 437 | 97 | 439 | 98 |
| Affective disorder | 334 | 74 | 321 | 71 |
| Substance use disorder | 250 | 56 | 236 | 52 |
| Received services from Eastern State Hospital | 133 | 30 | 133 | 30 |
| ≥96 days in the state hospital in the two years before PACT enrollment | 263 | 58 | 262 | 58 |
| Monthly cost of state hospital use ($) | ||||
| Before PACT enrollment | 2,805±3,147 | 2,492±2,828 | ||
| After PACT enrollment | 1,458±3,180 | 2,838±4,453* | ||
| Monthly cost of all psychiatric stays ($) | ||||
| Before PACT enrollment | 3,222±3,165 | 2,877±2,950 | ||
| After PACT enrollment | 1,837±3,798 | 3,446±5,178* | ||
a
Means were tested with t tests with unequal variances, and proportions were tested with chi square tests.
*p<.01; no other significant differences were found.
At baseline, approximately one-fifth of the overall sample had used a state hospital in any month in the two previous years, and 58% met the criterion of “high users” (an average of ≥96 days in the previous two years). At baseline, average monthly state hospital costs for a PACT enrollee were $2,805. After PACT enrollment, the level of state hospital costs dropped to $1,458, whereas for consumers in the control group, costs increased to an average monthly costs of $2,838 (p<.01).
The effect of PACT on state hospital costs over the three-year follow-up period was previously estimated to be a reduction of $12,700 per person per year (or $1,058 per month), compared with the costs for consumers in the control group (14). The term “reduction” as used in this article reflects the difference in costs between PACT participants and the matched control group participants—not a reduction from baseline costs. As shown in Figure 1, the estimated effect of PACT actually varied substantially over time. The average monthly cost reductions during the initial quarter of PACT enrollment were almost $3,500 per person, nearly triple the three-year average. By the second quarter after PACT enrollment (months 4–6), the differential costs decreased to $2,790 per person per month. For each subsequent quarter of PACT enrollment, a decline in differential state hospital costs was observed for the PACT participants compared with the control group. By the seventh quarter (months 19–21), the difference between PACT and control participants had flattened out, but PACT participants continued to have lower state hospital costs than control group participants. Only in the last quarters (months 28–33) were the state hospital costs virtually indistinguishable between the two groups.
Figure 1 Estimated differences in per-person state hospital costs between PACT and control group participants, by postenrollment quartera
aPACT, recovery-oriented assertive community treatment. Positive dollar values indicate reductions in costs (costs for control participants minus costs for PACT participants). Bars represent 95% confidence intervals.
We examined whether the time trends in state hospital costs varied according to baseline measures of use. As shown in Figure 2, state hospital cost reductions were much larger for PACT participants with high baseline levels of state hospital use, and these differences persisted for the full study period. In contrast, PACT participants who did not have at least 96 days in the state hospital before PACT enrollment showed a reduction in state hospital costs only for the first three quarters; beginning in the fourth quarter (month 10 postenrollment), state hospital costs for PACT participants did not differ significantly from state hospital costs for the control group participants.
Figure 2 Estimated differences in per-person state hospital costs between PACT and control group participants, by high and low baseline state hospital use and postenrollment quartera
aPACT, recovery-oriented assertive community treatment. Positive dollar values indicate reductions in costs (costs for control participants minus costs for PACT participants). Bars represent 95% confidence intervals.
State hospital costs accounted for the bulk of composite psychiatric stay costs, because of the low use of local hospitals and crisis stabilization units. Therefore, trends in psychiatric stay costs (Figures 3 and 4) were similar to those for the state hospital, except that the lack of difference between PACT and control group participants occurred earlier (third quarter, or month 7) for those with lower state hospital use at baseline.
Figure 3 Estimated differences in per-person costs on a composite measure of psychiatric stays between PACT and control group participants, by postenrollment quartera
aPACT, recovery-oriented assertive community treatment. The composite measure reflects state and local hospital costs and costs for stays on a local crisis stabilization unit. Positive dollar values indicate reductions in costs (costs for control participants minus costs for PACT participants). Bars represent 95% confidence intervals.
Figure 4 Estimated differences in per-person costs on a composite measure of psychiatric stays between PACT and control group participants, by high and low baseline state hospital use and postenrollment quartera
aPACT, recovery-oriented assertive community treatment. The composite measure reflects state and local hospital costs and costs for stays on a local crisis stabilization unit. Positive dollar values indicate reductions in costs (costs for control participants minus costs for PACT participants). Bars represent 95% confidence intervals.
We also investigated learning among ACT teams during the 33-month postenrollment period. We found no difference in state hospital or composite psychiatric stay costs by time enrolled in PACT (data not shown).
Discussion
Consistent with our previous findings (14), this study indicated that the timing of PACT effects on state psychiatric hospital costs depended on the PACT participant’s level of state hospital use at baseline. For high users, the biggest effect occurred in the first quarter after PACT enrollment; however, reductions were observed throughout the entire postenrollment period. In contrast, for low users at baseline, no reductions occurred after the third quarter. These results raise a number of programmatic and policy issues. Results for a composite measure of psychiatric stay costs were largely consistent with results for state hospital costs, indicating that large offsets were not occurring from increases in psychiatric stays at local hospitals or crisis stabilization units.
These results indicate that the largest gain from PACT occurred initially and tapered off over time. Reductions in state hospital use occurred largely among consumers with a history of high levels of state hospital use, which is consistent with other literature. In our study, the reductions in state hospital costs for PACT participants compared with control group participants leveled out at just over $1,500 per month. It remains a limitation of our analysis that we were unable to determine the effect of PACT on outpatient costs or other related outcomes. Currently, Washington State spends $10.4 million per year for up to 800 consumers who receive PACT team services, averaging approximately $1,100 per person per month. For participants with high state hospital use, PACT is clearly paying for itself, at least in terms of reductions in the types of costs considered in this study. The lack of cost data on outpatient services limits us from offering a more precise estimate of total cost reductions over consumers in the control group who did not receive PACT services. However, the high intensity of service use within PACT teams suggests that any costs associated with external outpatient services would likely be nominal. For PACT enrollees with low baseline levels of state hospital use, reductions in composite psychiatric costs occurred only for the first six months on PACT.
Our results suggest that it may be useful to consider a much shorter enrollment period in PACT instead of the traditional “ACT-for-life” standard. Further extensions of PACT services could then be reauthorized in time-limited increments. Although the original ACT model allowed for titrated services depending on need, it did not consider adjusted reimbursements. One approach to adjusted reimbursements, which has taken hold in Holland (20,21), is to enroll consumers with severe mental illness in regular case management services and then ramp up the intensity of services to the level of ACT when a person requires it. Yet another option is exemplified by New Mexico’s Medicaid program, which authorizes a two-tiered reimbursement structure for ACT teams—a higher rate for consumers needing intensive services and a lower rate for those needing less frequent staff contact (9).
The optimum time on an ACT team caseload, however, needs to be considered in the context of outcomes other than decreased hospitalization. Research must address the question of whether the benefits that accrue from receiving ACT services change over time, with initial benefits from decreased hospitalization and later benefits from improvements in functional and quality-of-life outcomes. The lack of information about functional and quality-of-life outcomes is a limitation of the administrative data used in this study that should be noted.
Further research that focuses on the occurrence and timing of ACT’s effects on promoting community adjustment and recovery—rather than just avoiding negative outcomes such as frequent or prolonged hospitalization—can help to clarify these issues. Comparative studies of recovery-oriented ACT and traditional ACT would help to determine whether the rapid timing of reductions in state hospital use, as reported here for Washington State’s PACT teams, can be attributed to recovery-oriented practices and associated opportunities for consumers of personal growth and independence. The measurement and timing of recovery and whether ACT teams have any particular advantage over other service interventions should also receive priority attention in future studies.
Conclusions
Washington State’s PACT teams had their greatest effect in reducing state and local psychiatric hospital use immediately after consumer enrollment, and these effects tapered off over time. Further research is needed to address other positive functions of ACT and their contributions to the timing and sustainability of consumer recovery. In this sense, it would be premature to conclude that ACT treatment should be terminated when the cost reductions from decreases in hospitalization become negligible.
Acknowledgments and disclosures
This research was supported in part by the Substance Abuse and Mental Health Services Administration (SAMHSA), through a Mental Health Services Transformation Grant to Washington State. Assistance from staff of the Washington State Mental Health Transformation Project (in particular, Ron Jemelka, Ph.D.) and the Research and Data Analysis Division of the Washington State Department of Social and Health Services (in particular, David Mancuso, Ph.D.) is gratefully acknowledged. Shirley Richards, B.S., provided excellent programming support. The findings and interpretations reported above are those of the authors and do not imply endorsement by SAMHSA, the Washington State Department of Social and Health Services, or the collaborating universities.
Dr. Domino has received research funding for an unrelated project from Eli Lilly and Company. The other authors report no competing interests.
References
1.
Witheridge TF: The “active ingredients” of assertive outreach. New Directions for Mental Health Services 52:47–64, 1991
2.
Allness DJ: The Program of Assertive Community Treatment (PACT): the model and its replication. New Directions for Mental Health Services 74:17–26, 1997
3.
Stein LI, Test MA: Alternative to mental hospital treatment: I. conceptual model, treatment program, and clinical evaluation. Archives of General Psychiatry 37:392–397, 1980
4.
McRae J, Higgins M, Lycan C, et al.: What happens to patients after five years of intensive case management stops? Hospital and Community Psychiatry 41:175–179, 1990
5.
Hackman AL, Stowell KR: Transitioning clients from assertive community treatment to traditional mental health services. Community Mental Health Journal 45:1–5, 2009
6.
Salyers MP, Masterton TW, Fekete DM, et al.: Transferring clients from intensive case management: impact on client functioning. American Journal of Orthopsychiatry 68:233–245, 1998
7.
Dincin J, Pinkerton M, Rychlik K, et al.: Impact of flexible duration assertive community treatment: program utilization patterns and state hospital use. Journal of Rehabilitation 65:25–30, 1999
8.
Rosenheck RA, Dennis D: Time-limited assertive community treatment for homeless persons with severe mental illness. Archives of General Psychiatry 58:1073–1080, 2001
9.
New Mexico Interagency Behavioral Health Service Requirements and Utilization Guidelines: Assertive Community Treatment. Santa Fe, New Mexico Behavioral Health Collaborative, May 2010. Available at http://www.bhc.state.nm.us/pdf/H0039%20(5.20.10).pdf
10.
Kasprow WJ, Rosenheck RA: Outcomes of critical time intervention case management of homeless veterans after psychiatric hospitalization. Psychiatric Services 58:929–935, 2007
11.
Dixon L, Goldberg R, Iannone V, et al.: Use of a critical time intervention to promote continuity of care after psychiatric inpatient hospitalization. Psychiatric Services 60:451–458, 2009
12.
Herman DB, Conover S, Gorroochurn P, et al.: Randomized trial of critical time intervention to prevent homelessness after hospital discharge. Psychiatric Services 62:713–719, 2011
13.
Draine J, Herman DB: Critical time intervention for reentry from prison for persons with mental illness. Psychiatric Services 58:1577–1581, 2007
14.
Morrissey JP, Domino ME, Cuddeback GS: Assessing the effectiveness of recovery-oriented ACT in reducing state psychiatric hospital use. Psychiatric Services 64:303–311, 2013
15.
Bjorklund RW, Monroe-Devita M, Reed D, et al.: Washington State’s initiative to disseminate and implement high-fidelity ACT teams. Psychiatric Services 60:24–27, 2009
16.
Client Counts and Service Costs. Olympia, Washington State Department of Social and Health Services, Research and Data Analysis Division. Available at clientdata.rda.dshs.wa.gov. Accessed Sept 2012
17.
Estee S, Wickizer T, He L, et al.: Evaluation of the Washington state screening, brief intervention, and referral to treatment project: cost outcomes for Medicaid patients screened in hospital emergency departments. Medical Care 48:18–24, 2010
18.
Basu A: Extended generalized linear models: Simultaneous estimation of flexible link and variance functions. Stata Journal 5:501–516, 2005
19.
Dieterich M, Irving CB, Park B, et al.: Intensive case management for severe mental illness. Cochrane Database of Systematic Reviews 10: CD007906, 2010
20.
van Veldhuizen JR: FACT: a Dutch version of ACT. Community Mental Health Journal 43:421–433, 2007
21.
Drukker M, van Os J, Sytema S, et al.: Function assertive community treatment (FACT) and psychiatric service use in patients diagnosed with severe mental illness. Epidemiology and Psychiatric Sciences 20:273–278, 2011
Information & Authors
Information
Published In
Cover: Geraldine Lee #2, by George Wesley Bellows, 1914. Oil on panel, 38 × 30 inches. The Butler Institute of American Art, Youngstown, Ohio.
History
Published in print: April 2013
Published online: 15 October 2014
Authors
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBGet Access
Login options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).