Comparison of Outcomes of Acute Care in Short-Term Residential Treatment and Psychiatric Hospital Settings
Abstract
OBJECTIVE: The study compared the demographic and diagnostic characteristics of clients and the outcomes of treatment in five short-term acute residential treatment programs and two acute hospital-based psychiatric programs. METHODS: A total of 368 clients in the short-term acute residential treatment programs and 186 clients in the psychiatric hospital programs participated in an observational study. The study used a repeated-measures design and assessed participants on multiple standardized measures of symptoms and functioning at admission, discharge, and four-month follow-up. Comparisons between the two groups were conducted separately by diagnostic category. Measures included the Brief Symptom Inventory, the Behavior and Symptom Identification Scale-32, the Medical Outcomes Short-Form-36, and the Client Satisfaction Questionnaire-8. RESULTS: The two types of programs admit persons with similar levels of acute distress who have comparable levels of improvement at discharge and an equivalent degree of short-term stability of treatment gains. Costs of treatment episodes were considerably lower for the short-term residential programs, and client satisfaction with the two types of programs was comparable. CONCLUSIONS: Short-term acute residential treatment is a less costly yet similarly effective alternative to psychiatric hospitalization for many voluntary adult patients.
Public and private behavioral health care systems are struggling with the pressure to contain or reduce costs while simultaneously maintaining or improving the quality of care. A logical outgrowth of this struggle is research on treatment outcomes that attempts to address fundamental questions about what a particular treatment is accomplishing and whether it is worth the cost (1,2,3). The measurement of outcomes becomes especially important as publicly funded systems implement various forms of managed care. In this context, it is important to document effective, yet less costly, alternative models for providing needed services (4,5).
The study reported here focused on five publicly funded short-term acute residential treatment facilities located in San Diego County, California (6). Operated by Community Research Foundation and funded by San Diego County Mental Health Services, these 11- to 14-bed facilities are designed to provide a less costly alternative to hospitalization for adults who need acute psychiatric care but do not require restraint or a locked facility. They offer brief, intensive, milieu-based treatment with a psychosocial rehabilitation orientation in a homelike neighborhood setting.
Several features are reported to distinguish the short-term acute residential treatment facilities in San Diego from similar programs in other locales. The San Diego programs admit clients who would otherwise be at high risk for acute psychiatric hospitalization. For example, clients who are admitted to the programs may be experiencing intense suicidality or psychosis. About half of the admissions have comorbid substance-related diagnoses. Medical complications are also frequently present. Most staff members have master's- or doctoral-level training, and psychiatric and nursing care is available on site. Further, the treatment is client centered, focusing on the individual needs of each client. The homelike setting makes possible the inclusion of household chores and meal preparation as components of the treatment experience. Although most activity would be characterized as milieu based, clients also have frequent personal contact with staff in daily individual and group sessions. Two staff members are present during overnight shifts. Daytime staffing typically includes at least one staff member for every three to four clients.
The comparison facilities consisted of two acute psychiatric hospital programs located in San Diego County. The programs were accredited by the Joint Commission on Accreditation of Healthcare Organizations. Identifying information related to the hospitals has been withheld due to agreements for anonymity with both hospital programs.
The main goals of the study were to compare the two types of programs on the following dimensions: the severity of clients' disturbance at the time of admission to the program, the outcome of treatment, the length of stay, the cost per treatment episode, and clients' satisfaction with services.
Methods
The study was observational and used a repeated-measures design. We compared characteristics of clients from the five residential programs and the two hospital programs at admission, discharge, and four months after discharge. The research protocol received the approval of the institutional review boards with jurisdiction over each program. Data from each type of program were combined, and the combined data for each type were compared.
To increase the study's validity, multiple standardized measures, each with subscales addressing various dimensions of symptoms and functioning, were used. The Brief Symptom Inventory (BSI) (7) was administered at admission only and the Client Satisfaction Questionnaire (CSQ-8) (8,9) at discharge only. The Behavior and Symptom Identification Scale (BASIS-32) (10) and the Medical Outcomes Study Short Form-36 (SF-36) (11) were administered at admission, discharge, and four-month follow-up. The self-report measures were supplemented by information from charts and computerized information systems, including reports of diagnoses and previous admissions.
Participants were eligible for the study if they spoke English, did not have a substance-related primary diagnosis, and signed a consent form. Adults consecutively admitted to each facility who met participant criteria were approached for participation in the study. Within 48 hours of admission and discharge, participants completed a battery of self-report measures. Follow-up interviews were conducted over the telephone by highly trained staff. The study period, including the follow-up interviews, spanned the period from August 1993 to December 1995.
The analysis used demographic and diagnostic data and mean subscale scores on the BSI, BASIS-32, SF-36, and CSQ-8. Demographic variables included age, employment status, ethnicity, gender, and marital status.
The primary analyses examined the severity of clients' disturbance at the time of admission; the outcome of treatment, defined as change between admission, discharge, and follow-up; the length of stay; the episode cost; and satisfaction with services received. As multiple statistical tests were being conducted, results were held to the more stringent alpha level of p<.01 for statistical significance. Because the primary diagnostic category appeared to have clinically relevant implications, data for study participants in specific diagnostic groups were analyzed and reported separately.
Results
Study participants
A total of 554 clients participated in the study at program admission and discharge, 368 from the short-term acute residential treatment facilities (53 percent of admissions during the study period) and 186 from the hospital-based psychiatric programs (20 percent of admissions during the study period). A higher proportion of clients from both groups—75 percent of the clients admitted to the residential programs and 42 percent of those admitted to the hospital programs—agreed to participate and signed a consent form at admission. However, not all of those clients were represented in data collected at discharge; 16 percent of the clients admitted to the residential programs and 14 percent of those admitted to the hospital programs were discharged without completing the discharge testing, and 6 percent of the residential clients and 8 percent of the hospital clients withdrew consent.
Follow-up interviews were originally planned for three months after discharge, but due to administrative delays and difficulty locating participants, the actual time elapsed from the day of discharge to the follow-up contact averaged four months (mean± SD=127.8±11.7 days). Follow-up interviews were successfully completed with 162 study participants from the residential programs (44 percent) and 89 participants from the hospital programs (48 percent).
Primary DSM-IV diagnoses were grouped into primary diagnostic categories including major depression, psychosis (including schizophrenia and schizoaffective and other psychotic disorders), bipolar disorder, other depression, and other diagnoses.
To reduce variability, the 36 participants with diagnoses in the "other" category—24 from the residential programs and 12 from the hospital programs—were excluded from the analysis. The analysis thus included data collected at admission and discharge for the remaining 344 study participants from the residential programs, 150 of whom (44 percent) participated in the follow-up interviews, and 174 participants from the hospital programs, 82 of whom (47 percent) participated in the follow-up interviews.
The representativeness of participants was explored in an analysis using data from administrative databases. Demographic and diagnostic data and length of stay of clients who participated and those who did not were compared. Findings suggested a bias against inclusion of some of the most seriously disturbed clients in both treatment settings.
In both settings, a lower proportion of clients with psychotic diagnoses and a higher proportion of clients with major depression were found among the study participants than among the nonparticipants. In the residential programs, 14 percent of the participants had a psychotic diagnosis, compared with 29 percent of the nonparticipants, and 55 percent of the participants had major depression, compared with 37 of the nonparticipants. In the hospital programs, 23 percent of the participants and 49 percent of the nonparticipants had a psychotic diagnosis, and 41 percent of the participants and 27 percent of the nonparticipants had major depression.
Interviews with clinical staff at the study sites corroborated the perception that severely decompensated, oppositional, or paranoid clients were less likely to participate. Although study participants were not fully representative of the populations treated in these programs, the biases operating for and against inclusion appeared to have been consistent across treatment settings and therefore did not preclude comparison.
As more than half of the participants in both study groups could not be located at follow-up, we compared outcome measures and demographic and diagnostic data of the participants we were able to contact with those of the participants who were not contacted. We found little evidence of a selection bias through attrition. The residential program clients who participated in the follow-up included proportionally more female clients than the group lost to follow-up (58.7 percent, compared with 40.7 percent, χ2=10.96, df=1, p<.01). However, this difference was not related to outcome. Among the clients in the hospital programs, we found no significant differences between the participants lost to follow-up and those we located.
Comparability of participants
The groups from the residential programs and the hospital programs were compared on demographic and diagnostic characteristics and mean scores on the depression, paranoia, psychosis, and global severity subscales of the BSI at the time of admission. The results are shown in Table 1.
We found significant differences between the two groups on diagnostic categories, gender, marital status, and employment. To assess clinical relevance, we examined the effect of these factors on the mean admission scores on the overall scale of the BASIS-32 for each of the groups. The BASIS-32 overall scale was selected because it is a composite scale that summarizes information from all five subscales of the BASIS-32. We found a significant effect for primary diagnostic category among the participant group from the residential programs (F=3.76, df=3,330, p=.01). No other significant effects were noted.
Severity of disturbance at admission
Analysis of variance (ANOVA) procedures were performed for each diagnostic group using mean scores at admission on all subscales of the BASIS-32 and the emotional well-being, emotional role functioning, and social functioning scales of the SF-36.
Although the mean scores of the clients with major depression and psychotic diagnoses indicated slightly more disturbance for participants in the residential programs, the differences between treatment settings were not statistically significant. Similarly, the clients with bipolar disorder in the residential programs showed slightly more severe symptoms on most measures at admission, but only the difference in scores on the social functioning scale was statistically significant (mean±SD=18.9±17.2 for the residential program clients, compared with 32.9±22.7 for the hospital program clients; F=7.44, df=1,59, p<.01). The magnitude-of-effect estimates, which assess the strength of an association as opposed to its statistical significance (12), supported reporting this difference as meaningful (eta2=.11). Mean scores for study participants with other depressive diagnoses were quite similar for the residential program and hospital groups.
Although significant differences were found between the two groups on some demographic and baseline measures, separating the results into diagnostic categories eliminated the differences for all but the scores on the social functioning subscale of the SF-36 for participants with bipolar disorder. These findings suggested that the residential program clients and hospital program clients in separate diagnostic categories were satisfactory comparison groups.
Treatment outcomeand stability
Outcome was assessed by repeated-measures ANOVA procedures using six subscales of the BASIS-32 and three subscales of the SF-36 for each diagnostic category. The BASIS-32 subscales included relation to self and others, daily living skills, depression-anxiety, impulsive-addictive behavior, psychosis, and the overall scale, which summarizes the other subscales. The emotional well-being, emotional role functioning, and social functioning subscales of the SF-36 were selected, as they are most closely related to mental health issues.
Among clients with major depression, follow-up data were available for 89 of the 199 study participants from the residential programs, or 45 percent, and for 48 of the 85 participants from the hospital programs, or 56 percent. Repeated-measures ANOVA showed significant differences between groups on the daily living skills subscale of the BASIS-32 (F=5.36, df=2,268, p<.01), but not on other BASIS-32 subscales or on the SF-36 subscales. Clients in the residential programs with major depression reported higher levels of distress at admission than those in the hospital programs (mean score of 2.6, compared with 2.4 on the daily living skills subscale of the BASIS-32) and slightly more improvement at discharge (mean score=1.6, compared with 1.8). At follow-up, clients with major depression who had been discharged from the residential programs reported a slight improvement, and clients from the hospital programs reported a slight worsening (mean score=1.5 for the residential programs, compared with 1.9 for the hospital programs). The magnitude-of-effect estimates for these comparisons was moderate (eta2=.08).
Fifty-four clients from the residential programs and 53 clients from the hospital programs had a psychotic diagnosis. Follow-up data were available for 24 of the residential clients, or 44 percent, and 20 of the hospital clients, or 38 percent. We found no significant differences between the groups.
There were 61 participants in the bipolar disorder group and 66 in the group with other depressive diagnoses. However, follow-up data were available for only seven subjects in each of these categories, limiting our analysis to a comparison of mean admission and discharge scores. No significant differences in these scores between clients from the two types of treatment settings were found in either diagnostic category.
The effects of attrition on outcome were examined by repeated-measures ANOVA procedures comparing residential program clients and hospital clients in each diagnostic group at admission and discharge only. Excluding the follow-up data more than doubled the number of participants in the analysis. Participants from both types of settings and in all diagnostic groups demonstrated robust and significant improvements at discharge (p<.01) on all outcome measures except the emotional role functioning scale of the SF-36 for participants with major depression. However, no significant differences were found between the residential program clients and the hospital program clients.
Length of stay
Independent-samples t tests were used to compare the mean length of stay of clients from the two types of programs in each of the diagnostic groups. The difference between groups for clients with major depression was not significant. For clients with a psychotic diagnosis or with bipolar disorder, the mean length of stay for those treated in the residential programs was longer than for those treated in the hospital programs (12.7 days, compared with 10.1 days for the participants with psychotic disorders; t=1.84, df=105, p=.05; and 14.5 days, compared with 9.2 days for the participants with bipolar disorder; t=2.93, df=59, p<.01). For study participants with other depressive disorders, the difference in length of stay between the residential programs and the hospital programs was not statistically significant.
Cost of treatment episodes
The Medi-Cal (California Medicaid) reimbursement rate in San Diego County at the time of the study was $186 per person per day for short-term acute residential treatment programs and $395 per day for psychiatric hospital programs. Although not all study participants had their care funded in this manner, these standardized rates were used to approximate costs for comparison. Per-episode costs were calculated by multiplying the daily reimbursement rate for each program type by the actual length of stay for each study participant by diagnostic group.
The mean episode costs for the different diagnostic groups are presented in Table 2. The differences in program costs are even more striking because the residential program costs included costs for psychiatric, pharmacy, and laboratory services, while the hospital costs did not.
Client satisfaction
The CSQ-8, which was administered at discharge, comprises eight qualitative questions that probe areas related to the person's experience of receiving treatment. Mean scores of study participants in the residential programs and the hospital programs were compared using independent-samples t tests for each diagnostic group. There were no significant differences between the mean scores of the two groups in any diagnostic category except major depression.
Residential program clients with major depression reported significantly higher levels of satisfaction than hospital program clients with that diagnosis on two questions. On the question of how they would rate the quality of service received, residential clients with major depression had a mean score, on a scale from 1 to 4, of 3.6, compared with 3.4 for hospital clients with major depression (t=2.51, df=118, p=.01). On the question of whether the services helped them deal more effectively with their problems, residential clients with major depression had a mean score of 3.6, compared with 3.3 for hospital clients with major depression (t=2.85, df=119, p<.01). However, in both cases, the differences, although significant, were relatively small. Participants from both types of programs consistently reported mean levels of satisfaction that fell into the upper 25 percent of the response range.
Readmissions
We collected self-report information about readmissions to any hospital or residential program during the follow-up period. Twenty residential program clients, or 13 percent, and 19 hospital program clients, or 23 percent, reported at least one admission to a hospital during the follow-up period. One participant from the hospital programs, or 1 percent, and 21 participants from the residential programs, or 14 percent, reported readmissions to a residential program, and five participants reported readmissions to both a hospital program and a residential program during the follow-up period.
If we combine the reported number of readmissions to the residential programs and the hospital programs for a total number of acute care readmissions, 27 percent of the residential clients and 24 percent of the hospital clients reported acute care readmissions during the follow-up period, not a significant difference.
Discussion and conclusions
This study did not have an experimental design and thus involved no manipulation of study participants or experimental procedures. The study simply attempted to measure and document what occurred at admission, discharge, and four months after discharge in a naturalistic manner, using primarily self-report data. Each hospital and short-term acute residential treatment facility implemented a single-group repeated-measures design in which study participants served as their own control subjects. The results were then combined and compared.
Of the nine clinical outcome measures examined across four diagnostic categories, significant differences were found only on the daily living skills subscale of the BASIS-32 for the major depression group, with participants in the residential programs reporting slightly better outcomes. Perhaps a more important finding is the striking consistency of similarities in the results at admission, discharge, and follow-up between the residential clients and the hospital clients. The findings suggest that, compared with hospital programs, the residential treatment facilities admitted persons with similar levels of acute distress who demonstrated comparable improvements with treatment and similar short-term stability of treatment gains within the diagnostic groups considered. Because the cost of residential treatment is about half that of hospital treatment, the results of this study suggest that short-term acute residential treatment programs provide a cost-effective alternative to acute psychiatric hospitalization for many adult clients within these diagnostic groups who do not require restraint or a locked facility.
Limitations
Repeated-measures designs in which study participants serve as their own control subjects are cost-effective and relatively unobtrusive, but this approach imposes limitations on the conclusions that may be drawn. In addition, although it is important to ascertain the stability of treatment gains by assessing study participants at follow-up, their status at follow-up after receiving acute care services, especially after longer time periods such as four months, may not be attributable to factors connected with the treatment.
This study's generalizability was also limited by a lack of representativeness in the sample. However, the biases were consistent across treatment settings, therefore allowing comparisons between the residential programs and the hospital programs.
Future research
Defining and measuring the outcome of behavioral health care services, especially for persons with severe and persistent disorders, is a complex task (3). The authors intend to use the study reported here as a pilot study for a controlled randomized comparison. Despite researchers' efforts in this direction over the years (13,14,15,16,17,18), well-designed studies to provide empirical support for alternative forms of acute psychiatric treatment continue to be needed (4). In addition, distinctions need to be made among the range of programs calling themselves alternatives to acute psychiatric hospitalization and among the clientele they serve so that accurate comparisons of these programs can be made.
Another logical step is the analysis of differential outcomes (19,20), with the aim of determining which persons experiencing an acute psychiatric crisis are best treated in a hospital alternative and which are best treated in a hospital setting.
A last perplexing issue is the continuing slow pace with which alternatives to acute psychiatric hospitalization have been adopted. Various authors have commented on the failure of hospital alternatives to become more widely accepted (21,22,23). Mosher's query (22) of more than a decade ago— "Why has research failed to be translated into practice?"—continues to be relevant. In the late 1990s, fiscal pressures and an emphasis on verifiable treatment outcomes in the behavioral health care field are combining to bring another wave of attention to cost-effective treatment alternatives. Efforts need to be directed toward the adoption of valuable and cost-effective alternate treatments that have been shown to provide comparable or improved treatment outcomes.
Acknowledgments
The study was funded by San Diego County Mental Health Services, Community Research Foundation, and grants from the Vista Hill Foundation in San Diego.
1 Scores range from 27 to 69, with higher scores indicating more severe symptoms.
2 Significant difference between groups; χ2=18.33, df=3, p<.01
3 Percentages based on data for participants for whom secondary diagnoses were reported: 318 participants from the residential programs and 80 participants from the hospital programs
4 Significant difference between groups; χ2=9.31, df=1, p<.01
5 Significant difference between groups; χ2=5.96, df=1, p=.01
6 Significant difference between groups; χ2=68.25, df=2, p<.01
Footnote
Dr. Hawthorne is executive director of the Community Research Foundation, 444 Camino del Rio South, Suite 219, San Diego, California 92108 (e-mail, [email protected]). He is also associate clinical professor in the department of psychiatry at the University of California, San Diego. Dr. Green is research coordinator at Community Research Foundation and in private practice in San Diego. Dr. Lohr is chief of the psychiatry service at the Veterans Affairs Medical Center in San Diego. Dr. Hough is codirector for research on child and adolescent mental health services at Children's Hospital and Health Center in San Diego. Dr. Smith is chief of program review and development at San Diego County Mental Health Services. Earlier versions of this paper were presented at poster sessions of the annual meeting of the American Public Health Association held November 9-13, 1997, in Indianapolis, at the Institute on Psychiatric Services held October 18-22, 1996, in Chicago, and at the annual convention of the American Psychological Association held August 8-12, 1996, in Toronto, Ontario.
References
1.
Epstein AM: The outcomes movement: will it get us where we want to go? New England Journal of Medicine 323:266-269, 1990
2.
Mirin SM, Namerow MJ: Why study treatment outcome? in Psychiatric Treatment: Advances in Outcome Research. Edited by Mirin SM, Gossett J, Grob MC. Washington, DC, American Psychiatric Press, 1991
3.
Rosenblatt A, Attkisson CC: Assessing outcomes for sufferers of severe mental disorder: a conceptual framework and review. Evaluation and Program Planning 16:347- 363, 1993
4.
Warner R (ed): Alternatives to the Hospital for Acute Psychiatric Treatment. Washington, DC, American Psychiatric Press, 1995
5.
Hoge MA, Davidson L, Sledge WH: Alternatives to acute hospitalization, in Managed Mental Health Care in the Public Sector: A Survival Manual. Edited by Minkoff K, Pollack D. Amsterdam, Harwood, 1997
6.
Brunton J, Hawthorne H: The acute non-hospital: a California model. Psychiatric Hospital 20:95-99, 1989
7.
Derogatis LR: The Brief Symptom Inventory (BSI): Administration, Scoring, and Procedures Manual: II, 2nd ed. Baltimore, Clinical Psychometric Research, 1992
8.
Attkisson CC, Zwick R: The Client Satisfaction Questionnaire: psychometric properties and correlations with service utilization and psychotherapy outcome. Evaluation and Program Planning 5:233-237, 1982
9.
Nguyen TD, Attkisson CC, Stegner BL: Assessment of patient satisfaction: development and refinement of a service evaluation questionnaire. Evaluation and Program Planning 6:299-314, 1983
10.
Eisen SV, Dill DL, Grob MC: Reliability and validity of a brief patient-report instrument for psychiatric outcome evaluation. Hospital and Community Psychiatry 45:242-247, 1994
11.
Ware JE, Snow KK, Kosinski M, et al: SF-36 Health Survey Manual and Interpretation Guide. Boston, New England Medical Center, Health Institute, 1993
12.
Tabachnik BG, Fidell LS: Using Multivariate Statistics, 2nd ed. New York, HarperCollins, 1989
13.
Hoult J, Reynolds I, Charbonneau-Powis M, et al: Psychiatric hospital versus community treatment: the results of a randomised trial. Australian and New Zealand Journal of Psychiatry 17:160-167, 1983
14.
Kiesler CA: Mental hospitals and alternative care: noninstitutionalization as potential pulic policy for mental patients. American Psychologist 37:349-360, 1982
15.
Mosher LR, Menn AZ: Community residential treatment for schizophrenia: two-year follow-up. Hospital and Community Psychiatry 29:715-723, 1978
16.
Stein LI, Test MA, Marx AJ: Alternative to the hospital: a controlled study. American Journal of Psychiatry 132:517-522, 1975
17.
Toprac MG: The Crisis Alternative Project: serious psychiatric crises can be safely and effectively treated outside the hospital. Impact, Summer 1995, pp 10-11
18.
Sledge WH, Tebes JK, Rakfeldt J, et al: Day hospital/crisis respite care versus inpatient care: part I. clinical outcomes. American Journal of Psychiatry 153:1065-1073, 1996
19.
Sheridan EP, Zuskar DM, Walsh SF, et al: Identifying variables predictive of success: the next step in alternatives to psychiatric hospitalization research. Journal of Community Psychology 17:356-368, 1989
20.
Herrell JM, Fenton W, Mosher LR, et al: Residential alternatives to hospitalization for patients with severe and persistent mental illness: should patients with comorbid substance abuse be excluded? Journal of Mental Health Administration 23:348- 355, 1996
21.
Friedman RS: Resistance to alternatives to hospitalization. Psychiatric Clinics of North America 8:471-482, 1985
22.
Mosher LR: Alternatives to psychiatric hospitalization: why has research failed to be translated into practice? New England Journal of Medicine 309:1579-1580, 1983
23.
Rissmeyer DJ: Crisis intervention alternatives to hospitalization: why so few? Psychosocial Rehabilitation Journal 9:54-63, 1985
Information & Authors
Information
Published In
History
Published online: 1 March 1999
Published in print: March 1999
Authors
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBGet Access
Login options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).