The Use of Standardized Discharge in IPS Supported Employment Programs

Using Mental Health Treatment Study (MHTS) data (9), we compared the quarterly probabilities of obtaining a first job for the IPS and control groups and determined the periods of initial unemployment during which IPS recipients continued to benefit from enrollment. The MHTS was a two-year RCT comparing the effectiveness of an intervention consisting of IPS, systematic medication management, and other behavioral services with treatment as usual (control) at 23 sites in the United States. Study participants included adult beneficiaries of Social Security Disability Insurance who had not worked during the 30 days prior to enrollment, had a primary diagnosis of schizophrenia or mood disorder, and lived within the catchment areas served by the study sites. The final analysis included 2,055 participants. Participants reported on their recent employment history every three months (9). For this time-to-event analysis, we defined an event as the first quarter during which a participant reported at least one day of competitive employment. All procedures contributing to this work complied with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.

Our analysis of the time to acquisition of a first job among recipients of the IPS intervention yielded both the survival function (overall proportion of the original sample obtaining a first job by a given quarter) and the quarterly probability of obtaining a first job (among those yet to find a first job). We used the Kaplan-Meier survival function in this analysis because this nonparametric method imposes no assumptions on the treatment-specific survival functions and the relationships between them. Naïve bootstrapped resampling of the intervention and control outcome data provided the estimates and confidence intervals. For each treatment condition, we conducted 1,000 bootstrapped resamplings of the same size as the observed data sets, computing, for each resampling, the Kaplan-Meier survival functions and corresponding quarterly probability of finding a first job among those who had yet to find a first job. We then randomly paired each of the 1,000 IPS and control resamplings and computed the quarterly difference in probability of finding a first job for each pair. To provide confidence intervals for estimates of probability by quarter of finding a first job and to test for the significance of the differences between those probabilities, we determined the 95% confidence intervals (CIs) for the treatment-specific probabilities and the differences in probability for each quarter based on each statistic’s observed distribution across 1,000 resampled data sets.

Figure 1 shows treatment-specific probabilities by quarter of finding a first job among those yet to find first jobs. Among recipients of the intervention, the median quarterly probability of finding a first job decreased from .15 (CI=.13–.18) in the first quarter to .04 (CI=.02–.06) in the last quarter. In the control group, the quarterly probability of finding a first job decreased from .10 (CI=.08–.12) to .02 (CI=.01–.03). Throughout the first 18 months of follow-up, IPS recipients were significantly more likely to find a first job than the control group. During this period, IPS recipients (N=1,004) found 493 first jobs, compared with 325 in the control group (N=1,051). Between the ninth and 18th months of follow-up, and despite a smaller sample size, IPS recipients found 62 more first jobs than did the control group. [A quarterly survival function and a table describing the quarterly probability of finding a first job in the original data set are available in an online supplement.]

Our analysis of the large-sample MHTS data showed that IPS recipients, compared with participants in the control group, were more likely to find a first job over 18 months. This finding suggests that making continued enrollment in IPS solely contingent on finding a first job within the first nine months would compromise the long-term employment outcomes of those discharged.

Natural attrition, the de facto means of discharge at most IPS programs, is based on individualized decisions made by recipients and their treatment teams. As such, it draws on the full range of available information, including recipient engagement, to determine continued enrollment status. Given the difficulty of studying the deliberative processes underlying natural discharge practices or the program-level retention patterns resulting from these practices, coupled with the need to make the most efficient use of the limited resources available to IPS programs, we must consider the utility of standardizing discharge policies based on objective measures of recipient activity prior to any widespread adoption. Basing recipient discharge on standardized conditions can guarantee a stable program-level enrollment and discharge dynamic, enable optimization in favor of a given outcome measure, and ensure that more people will receive IPS services. However, this approach prioritizes program efficiency over the outcomes of enrolled individuals and has no provision for other recipient characteristics to affect discharge decisions.

For example, a rule limiting participation in IPS to nine months disregards engagement level, ensuring both that disengaged recipients may remain enrolled for nine months and that engaged recipients who have yet to find a first job are discharged after nine months. In the MHTS, more than half of the nonworkers either had not engaged in IPS or disengaged during the study (10). Despite the inclusion of the disengaged recipients in the primary analysis, those enrolled in IPS continued to benefit relative to control participants for at least 18 months. Thus, a more nuanced approach to discharge, in which those who remain interested in working are retained in IPS, while others leave naturally, may be appropriate ethically and clinically.

Other real-world studies show that IPS specialists typically help clients to discontinue services or transfer support to case managers in much less time than 18 months (4). The extent to which natural attrition affects the cost-effectiveness of IPS compared with standardized discharge policies remains an open question. Also unknown is how well natural attrition, compared with standardized discharge policies, predicts the employment potential of IPS recipients.

Limitations of our study included the characteristics of the MHTS sample, the imposition of experimental conditions, and potential confounding of the effect of IPS by supporting services such as systematic medication management. Nevertheless, the MHTS included a large group of participants with significant geographic, ethnic, and racial diversity (9), and there is no evidence that the supportive services offered alongside IPS enhanced employment outcomes. We therefore conclude that our findings support continued reliance on clinical decision making as ethically defensible and, in the absence of contradictory data, the most efficient means of evaluating the continued enrollment of IPS recipients.

IPS recipients who do not find employment within nine months of enrollment may continue to benefit for several additional months. In real-world settings, IPS teams create new capacity naturally by allowing those who lose interest or who are not benefiting to drop out, at least temporarily, or, if they are working successfully, to discontinue IPS services and transfer support to case management. Although IPS programs must discharge some enrollees in order to provide services to additional recipients and improve cost-effectiveness, a Procrustean bed may foster disability rather than recovery. Further research into the potential gains in cost-effectiveness associated with a range of discharge practices, whether informal or standardized, will help IPS practitioners understand and minimize the impact of those practices on those who may not benefit as quickly from IPS.