Skip to main content

Abstract

Objective:

This study examined how accurately inpatient case managers predicted 30-day readmission and whether objective patient characteristics improved prediction accuracy.

Methods:

In this prospective study, inpatient case managers at a psychiatric hospital rated their concern (1, not concerned; 5, very concerned) about readmission after discharge of 282 privately insured patients. Sensitivity and specificity of the ratings were calculated. Logistic regression identified whether patient characteristics that could affect 30-day readmission improved prediction accuracy.

Results:

Concern levels ≥3 yielded 86% sensitivity, 37% specificity, and a positive predictive value (PPV) of 13%; levels ≥4 yielded 39% sensitivity, 78% specificity, and a PPV of 17%. Concern level independently predicted readmission; appointments within seven days postdischarge further improved model accuracy (p=.03) (area under the curve=.67, 95% confidence interval=.58–.78).

Conclusions:

Although not highly accurate, case manager concern identified some patients at higher risk of 30-day readmission. Appointments within seven days of discharge improved prediction accuracy.
Thirty-day hospital readmissions are increasingly a focus of health policy because they may reflect substandard care and are costly. Psychiatric populations have among the highest readmission rates (14), making treatment of mental illness a prime target for policy efforts aimed at reducing readmission. For these efforts to be successful, providers will need better tools to predict which patients are at higher risk.
Thirty-day readmission has been associated with prior hospitalizations (3,5,6), symptom acuity (69), co-occurring substance use disorders (3,9,10), medication nonadherence (10,11), decreased length of stay (3,9,12), and discharge planning (13). Research on the association between diagnosis and 30-day readmission has had mixed results (6,12). Less is understood about other potential influences on 30-day readmission, such as significant life stressors, family involvement, or elements of aftercare planning—including relationships between new patients and community providers and whether the aftercare plan is consistent with the recommendations of the inpatient team.
Easily implemented methods are needed to better predict who is at risk of 30-day readmissions. Clinician prediction is one possible method. A prior study found good specificity (86%) but low sensitivity (19%) for clinician predictions of three-month readmission of Medicaid patients with schizophrenia (14). However, prior research has not examined clinician accuracy in predicting readmissions of a diagnostically diverse psychiatric population or readmission within 30 days, both of which are highly relevant in the current policy environment. Also, prior literature does not address whether knowledge of objective patient characteristics can improve prediction accuracy.
We used a prospective survey of inpatient social workers and health plan data to answer the following questions: How accurately can inpatient social work case managers predict patient 30-day readmission? and Can objective patient information that is easily available to inpatient case managers improve the prediction’s accuracy? The overall goal of the study was to determine if information that could be readily obtained or utilized in usual care settings can assist providers and programs to better identify high-risk patients who need additional supports to prevent 30-day readmission.

Methods

This study received institutional review board approval. The study setting is a freestanding psychiatric hospital with six subspecialty inpatient units and a satellite site with a general psychiatry inpatient service. Between May 1, 2013, and August 31, 2013, inpatient social work case managers were asked to complete a brief survey for each discharged inpatient who was an enrollee of a private insurance health plan. The plan provided readmission utilization data for the patient during the 30-day postdischarge period. All participants provided written informed consent. The health plan utilization information was used to calculate prediction accuracy. The four-month observation window was selected on the basis of historical data on the hospital’s readmission rate for the health plan, and because a window of that length was anticipated to provide a sample size of readmitted patients that would provide at least 80% power for our analyses.
The survey consisted of two sections. Part 1 asked case managers, “On a scale from 1 to 5 below, please rate how strongly you have concerns that this patient may need a mental health or substance use disorder readmission within 30 days of discharge from here?” A response of 1 indicated not concerned or low risk, and a response of 5 indicated very concerned or high risk. Part 2 asked objective categorical questions about the patient that may be related to 30-day readmission, for example, questions suggested either by prior literature or by the clinical face validity of particular characteristics. These questions related to active substance use disorder, employment, homelessness, medication adherence, and family involvement during hospitalization. The survey also asked whether the team felt the discharge was premature, whether the aftercare plan included an appointment with a new outpatient clinic or provider, whether the aftercare plan was the preferred or first recommended choice, and whether the patient received an appointment with an outpatient provider within seven days of discharge. Respondents were instructed to leave items blank if they were not confident about the correct response.
We calculated sensitivity, specificity, and positive predictive value (PPV) of the case managers’ Likert-scale ratings to examine the optimal cutoff for predicting readmission. We conducted bivariate analyses of the case managers’ concern ratings (categorical variable) and responses to objective categorical questions in the survey (binary variable) to determine if any were associated with 30-day readmission. Variables with p values ≤.10 were included in a multivariable logistic regression model, with stepwise removal of explanatory variables no longer statistically significant. For low-frequency events, we used Firth’s penalized likelihood logistic regression. We calculated areas under the curve (AUC) receiver operating characteristics (ROC) for the final model. All analyses adjusted for potential clustering of observations within case managers and were conducted with SAS, version 9.2, and Stata, version 13.

Results

During this time period, 323 patients who were enrolled in the private health plan were discharged; surveys were completed for 87% (N=282). Thirty-four of 37 case managers (92%) participated. There was no difference in readmission rates between patients for whom surveys were or were not completed at discharge. Patients for whom surveys were completed had a mean±SD age of 40.5±17.3, and the greatest number were treated in the unit for detoxification of patients with co-occurring mental and substance use disorders (N=79, 28%). Others were treated in the units for depression and anxiety (N=62, 22%), psychotic disorders (N=14, 14%), geriatric psychiatry (N=25, 9%), and dissociative disorders (N=41, 15%) or in the general unit at the satellite site (N=36, 13%).
Twenty-eight patients (10%) had a behavioral health readmission within 30 days. Readmission rates for case manager concern levels of 3, 4, and 5 were 13.1%, 16.7%, and 17.6%, respectively. [A table displaying sensitivity and specificity for alternate cutoffs is available in an online supplement to this report.]
No patients had a repeat index admission during the study. In the unadjusted results, concern levels ≥3 resulted in sensitivity of 86%, specificity of 37%, and PPV of 13%. A cutoff score ≥4 resulted in sensitivity of 39%, specificity of 78%, and PPV of 17%.
In bivariate analyses, only student status and aftercare appointments within seven days of discharge were associated with readmission at the p<.10 level (Table 1). Responses for medication adherence were missing for 44 (16%) patients, but otherwise few data were missing.
TABLE 1. Characteristics of 282 patients discharged from a psychiatric hospital and association with 30-day readmission
 Overall sample (N=282)Not readmitted (N=254)Readmitted (N=28)
CharacteristicN responsesN%N%Association with readmission (p)N%Association with readmission (p)
Comorbid substance use disorder2791615814357ns1864ns
Employed2801234411244ns1139ns
Student2802692610<.1000<.10
Homeless280176146 311ns
New, significant stressor prior to hospitalization2791505413152ns1968ns
Medication adherence2381285411558ns1357ns
Family involved during hospitalization2802468822188ns2589ns
Patient discharged early280238218ns27ns
Aftercare plan included new outpatient provider or clinic2811956917870ns1761ns
Aftercare plan was preferred or recommended by team2812057318774ns1864ns
Aftercare appointment within 7 days of discharge2812569123593<.0012175<.001
The final multivariable model indicated that case manager concern level (odds ratio [OR]=1.52, 95% confidence interval [CI]=1.02–2.27, p=.04) and an appointment scheduled within seven days of discharge (OR=3.23, CI=1.16–9.00, p=.03) were associated with greater odds of 30-day readmission and had an AUC of .67 (CI=.58–78). Thus both variables were significant and independent predictors of 30-day readmission.

Discussion

Inpatient case managers’ prospective concern level was associated with greater odds of 30-day readmission, but it was not overall highly accurate. Still, our findings indicate that these predictions can be useful in identifying a significant proportion of patients at higher risk of 30-day readmission. In our population, with an approximately 10% readmission rate, a targeted intervention to reduce readmissions using a cutoff of ≥3 would mean that 65% of the sample would have received additional resources or attention to prevent readmission, including 86% of individuals who actually were readmitted. Using a cutoff ≥4 would mean that 24% of the sample—and 40% of the population that was actually readmitted—would have received additional resources.
A program with resource constraints may choose to implement two levels of intervention, depending on the patients’ cutoff scores. Patients who meet a lower cutoff for intervention, who represent a broader proportion of the at-risk population, might receive a postdischarge phone call, whereas patients who meet a higher cutoff threshold might receive more resource-intensive interventions, such as a home visit. Even if some high-risk patients are missed, identifying a significant proportion of higher-risk individuals could provide value.
Among the objective patient characteristics, only one—a follow-up visit scheduled for within seven days of discharge—was (negatively) associated with readmission. Not obtaining an appointment in the seven days after discharge may occur because of patient preferences—for example, patients with an ongoing relationship with a provider may prefer to schedule their own appointments or patients may be reluctant to schedule a follow-up visit in general—or practices by the outpatient clinic—for example, clinic personnel directly notify new patients about a discharge appointment or had no appointments available within seven days of discharge. Nevertheless, our results indicate that appointments not scheduled within seven days after discharge, regardless of the reason, are an independent risk of 30-day readmission. Research is needed to learn if improving scheduled appointment rates reduces 30-day readmission.
Several patient characteristics that have been associated with readmission in prior literature were not associated with readmission in this study. Possibly our study was underpowered to detect statistically significant associations between 30-day readmission and these characteristics. Consistent with prior literature, however, a comparison of the distribution of these characteristics among patients who were or were not readmitted suggests that there was an association in the expected direction between readmission and many characteristics, such as substance use disorders, employment status, homelessness, new or significant stressors, and new aftercare providers. Nonetheless, case manager predictions and scheduled follow-up within seven days of discharge were more strongly associated with readmission compared with these other characteristics. Possibly, that is because case manager concern levels incorporate a variety of risk and mitigating factors. Also, in contrast to prior literature (10,15), we found similar readmission rates for medication-adherent and -nonadherent patients. However, case managers skipped this question most frequently (16%), suggesting that they were unsure of the response. Possibly, this information was not missing at random, which may have affected our results.
A limitation was that although the survey included questions about patient characteristics that would be readily known by case managers, we did not verify this information with chart review, and it is possible that there were some inaccuracies in case managers’ responses. To minimize this possibility, we requested that the case managers avoid completing survey items if they were not confident of the response. The missing responses for medication adherence suggest that respondents followed these instructions.
Another limitation was that the study was conducted in a single hospital, which may limit generalizability. To mitigate this limitation, we included a diverse patient population from multiple inpatient units, including general and subspecialty units as well as an off-site satellite program. Further strengthening our findings was the high rate of case manager participation (92%) and survey completion (87%). Still, there may be hospital organizational characteristics that affected readmission prediction accuracy and limit the generalizability of our findings. For example, hospitals may vary in case managers’ ability to adequately interact with patients or family members (because of caseload constraints), which could impact their knowledge of pertinent patient history or characteristics and affect the accuracy of their concern level in predicting readmission.
Finally, although concern levels by inpatient case managers about 30-day readmission can be a very “user friendly” method to identify some patients at higher risk of readmission, we found few objective predictors of readmission. These results limit our ability to identify more systematic interventions that could be tailored to patients, other than scheduling appointments within seven days postdischarge. Future research is needed to test whether clinician ratings, coupled with an additional process that can help avoid rehospitalization for targeted patients, can successfully reduce readmissions. Such a process could include, for example, a clinical case conference to discuss patients who score above a 30-day readmission risk threshold. Similarly, our results may not generalize to interventions in which the same clinician who provides the ratings is given additional tasks on the basis of those ratings, for example, making additional postdischarge calls to patients, possibly without supplemental resources.

Conclusions

This study demonstrates that a simple Likert scale completed by inpatient social worker case managers can successfully identify a significant subset of patients who are at higher risk of 30-day readmission, albeit at the cost of misidentifying a relatively large proportion of patients who are not. Not having a scheduled follow-up within seven days of discharge improved the prediction accuracy of case manager concern level for 30-day readmission. Such information could be useful for targeting new interventions for patients determined to be at high risk, helping to reduce 30-day readmissions of patients with psychiatric illnesses.

Acknowledgments

The authors thank Optum for providing health plan readmission data for the study population.

Supplementary Material

File (appi.ps.201400282.ds001.pdf)

References

1.
Gilmer T, Hamblin A: Hospital Readmissions Among Medicaid Beneficiaries With Disabilities: Identifying Targets of Opportunity. Hamilton, NJ, Center for Health Care Strategies, 2010
2.
Jiang HJ, Wier LM: All-Cause Hospital Readmissions Among Non-Elderly Medicaid Patients, 2007. Rockville, Md, Agency for Health Care Research and Quality, 2010
3.
Mark TL, Tomic KS, Kowlessar N, et al: Hospital readmission among Medicaid patients with an index hospitalization for mental and/or substance use disorder. Journal of Behavioral Health Services and Research 40:207–221, 2013
4.
Hines AL, Barrett ML, Jiang HJ, et al: Conditions With the Largest Number of Adult Hospital Readmissions by Payer, 2011. Rockville, Md, Agency for Healthcare Research and Quality, 2014
5.
Schmutte T, Dunn CL, Sledge WH: Predicting time to readmission in patients with recent histories of recurrent psychiatric hospitalization: a matched-control survival analysis. Journal of Nervous and Mental Disease 198:860–863, 2010
6.
Swett C: Symptom severity and number of previous psychiatric admissions as predictors of readmission. Psychiatric Services 46:482–485, 1995
7.
Byrne SL, Hooke GR, Page AC: Readmission: a useful indicator of the quality of inpatient psychiatric care. Journal of Affective Disorders 126:206–213, 2010
8.
Lyons JS, O’Mahoney MT, Miller SI, et al: Predicting readmission to the psychiatric hospital in a managed care environment: implications for quality indicators. American Journal of Psychiatry 154:337–340, 1997
9.
Boaz TL, Becker MA, Andel R, et al: Risk factors for early readmission to acute care for persons with schizophrenia taking antipsychotic medications. Psychiatric Services 64:1225–1229, 2013
10.
Haywood TW, Kravitz HM, Grossman LS, et al: Predicting the “revolving door” phenomenon among patients with schizophrenic, schizoaffective, and affective disorders. American Journal of Psychiatry 152:856–861, 1995
11.
Bodén R, Brandt L, Kieler H, et al: Early non-adherence to medication and other risk factors for rehospitalization in schizophrenia and schizoaffective disorder. Schizophrenia Research 133:36–41, 2011
12.
Figueroa R, Harman J, Engberg J: Use of claims data to examine the impact of length of inpatient psychiatric stay on readmission rate. Psychiatric Services 55:560–565, 2004
13.
Steffen S, Kösters M, Becker T, et al: Discharge planning in mental health care: a systematic review of the recent literature. Acta Psychiatrica Scandinavica 120:1–9, 2009
14.
Olfson M, Mechanic D, Boyer CA, et al: Assessing clinical predictions of early rehospitalization in schizophrenia. Journal of Nervous and Mental Disease 187:721–729, 1999
15.
Sullivan G, Wells KB, Morgenstern H, et al: Identifying modifiable risk factors for rehospitalization: a case-control study of seriously mentally ill persons in Mississippi. American Journal of Psychiatry 152:1749–1756, 1995

Information & Authors

Information

Published In

Go to Psychiatric Services
Go to Psychiatric Services

Cover: Print table, by Frank Lloyd Wright (maker: William E. Nemmers), 1902–1903. White oak. Purchase, Emily Crane Chadbourne Bequest, 1972, the Metropolitan Museum of Art, New York City. Image copyright © The Metropolitan Museum of Art. Image source: Art Resource, New York City.

Psychiatric Services
Pages: 244 - 247
PubMed: 26522672

History

Received: 24 June 2014
Revision received: 11 May 2015
Accepted: 26 May 2015
Published online: 2 November 2015
Published in print: February 01, 2016

Authors

Details

Maya Reddy, L.I.C.S.W.
The authors are with the Department of Psychiatry, McLean Hospital, Belmont, Massachusetts (e-mail: [email protected]). Dr. Fitzmaurice is also with the Department of Biostatistics, Harvard School of Public Health, Boston. Dr. Busch is also with the Department of Psychiatry, Harvard Medical School, Boston.
Susan Schneiders-Rice, L.I.C.S.W.
The authors are with the Department of Psychiatry, McLean Hospital, Belmont, Massachusetts (e-mail: [email protected]). Dr. Fitzmaurice is also with the Department of Biostatistics, Harvard School of Public Health, Boston. Dr. Busch is also with the Department of Psychiatry, Harvard Medical School, Boston.
Casey Pierce, M.A.
The authors are with the Department of Psychiatry, McLean Hospital, Belmont, Massachusetts (e-mail: [email protected]). Dr. Fitzmaurice is also with the Department of Biostatistics, Harvard School of Public Health, Boston. Dr. Busch is also with the Department of Psychiatry, Harvard Medical School, Boston.
Garrett Fitzmaurice, Ph.D.
The authors are with the Department of Psychiatry, McLean Hospital, Belmont, Massachusetts (e-mail: [email protected]). Dr. Fitzmaurice is also with the Department of Biostatistics, Harvard School of Public Health, Boston. Dr. Busch is also with the Department of Psychiatry, Harvard Medical School, Boston.
Alisa Busch, M.D., M.S.
The authors are with the Department of Psychiatry, McLean Hospital, Belmont, Massachusetts (e-mail: [email protected]). Dr. Fitzmaurice is also with the Department of Biostatistics, Harvard School of Public Health, Boston. Dr. Busch is also with the Department of Psychiatry, Harvard Medical School, Boston.

Funding Information

Partners Psychiatry and Mental Health
The authors gratefully acknowledge the funding support of Partners HealthCare and Partners Psychiatry and Mental Health.The authors report no financial relationships with commercial interests.

Metrics & Citations

Metrics

Citations

Export Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

For more information or tips please see 'Downloading to a citation manager' in the Help menu.

Format
Citation style
Style
Copy to clipboard

View Options

View options

PDF/EPUB

View PDF/EPUB

Get Access

Login options

Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.

Personal login Institutional Login Open Athens login
Purchase Options

Purchase this article to access the full text.

PPV Articles - Psychiatric Services

PPV Articles - Psychiatric Services

Not a subscriber?

Subscribe Now / Learn More

PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.

Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).

Media

Figures

Other

Tables

Share

Share

Share article link

Share