Service Intensity as a Predictor of Competitive Employment in an Individual Placement and Support Model

Participants of the parent study had to be admitted to one of two day programs operated by Thresholds and be 18 or older. Participants met Illinois criteria for severe mental illness: a DSM-IV diagnosis of schizophrenia, other psychotic disorders, bipolar disorder, or obsessive-compulsive disorder and significant treatment history or significant functional impairments. Furthermore, participants were enrolled in Thresholds services for at least 30 days, had a goal of paid employment, and had not been competitively employment during the prior three months. Participants did not have a physical illness that would prevent their participation, agreed to exclusion from the nonassigned vocational program for two years, and attended two sessions of a research information group during the first 60 days after admission to Thresholds, after which they gave written consent to participate in the study.

A total of 194 persons were entered into the parent study (96 in the IPS arm and 98 in the diversified placement approach arm). The study presented here included participants who were randomly assigned to the IPS arm of the parent study, excluding five participants who discontinued IPS services within the first quarter, for a final sample of 91. The excluded group did not differ from the included sample on any individual characteristics (for example, demographic characteristics or clinical factors), did not receive any IPS contacts during the study period, and did not obtain employment before dropping out, but they did receive fewer mental health service hours during the two-year study period than the sample included in the study presented here (mean±SD=22.6±15.9 versus mean±SD=375.9±348.1; t=9.50, df=93, p<.001).

Data were entered into a software database, were double-checked for accuracy, and were analyzed using SPSS, version 11.0, for Windows and Hierarchical Linear Modeling, version 5.00, for Windows (21,22). We recorded zeros for IPS contacts for quarters after which participants dropped out of IPS services.

All hypotheses were examined in two manners. First, hypotheses were tested in a simple, aggregated fashion by totaling variables across the entire study and conducting hierarchical linear regressions. Second, data were examined longitudinally using hierarchical linear modeling (HLM) (23). HLM controls for the correlated nature of multiple observations made on the same individual and thus is appropriate for longitudinal data. In order to test for a potentially delayed effect of IPS services, a longitudinal, time-lag analysis was conducted. This analysis was identical to the other HLM analyses, but this analysis paired IPS contacts for a given quarter with the weeks worked in the following quarter. Zeros were imputed for IPS service contacts in the quarter before baseline.

In the parent study (17), 72 of 96 (75%) IPS participants obtained competitive employment; 45 of 96 (47%) worked at least 20 hours per week. Characteristics of the 91 participants in the study presented here can be found in Table 1. Within our study sample (N=91), 44 (48%) participants continued IPS services throughout the two-year study period (program continuers) but may not have had an IPS contact in each quarter. Dropouts were roughly evenly distributed across the study period. Program continuers did not differ from dropouts on demographic or clinical variables. We repeated all of the following analyses with participants enrolled in IPS throughout the study period program; results did not differ substantially, so results from the full study sample are reported below.

Table 2 shows service data, including IPS contacts for both the full sample and for those enrolled in services, for the entire follow-up period. Averages for quarters reflect the mean number of IPS contacts for participants still enrolled in IPS services; therefore averages exclude dropouts and participants with no data recorded. Participants averaged about six contacts per quarter in quarters in which they received at least one contact. In general, the number of IPS contacts declined across time (F=17.6, df=7 and 322, p<.001). IPS service receipt fell steadily from 86% of the sample in the first quarter to 42% in the final quarter. Over the two-year period, clients averaged 56.9 hours of mental health services per quarter. The hours of mental health services also declined with time (F=5.23, df=7 and 357, p<.001). Mental health service receipt fell from 100% in the first quarter to 59% in the final quarter. Intensity of IPS services was modestly positively correlated with mental health services (N=91, r=.25, p=.02).

Client background characteristics were generally unrelated to IPS service intensity (Table 1). (Continuous variables are not reported in Table 1, but those tested can be seen in Table 3.) The only exception was that fewer psychiatric hospitalizations in the year before the study were associated with more IPS contacts (N=91, r=–.20, p=.05). Length of hospital stay during the follow-up period was not associated with the number of IPS contacts.

Total IPS contacts for the full sample were positively associated with weeks worked (N=88, r=.30, p<.01). This correlation was not maintained when it was calculated for only the participants who obtained employment. A longitudinal analysis using HLM did not support an association between the number of IPS contacts in a given quarter and the weeks worked in the same quarter. However, the number of IPS contacts in a given quarter was positively associated with the number of weeks worked in the following quarter (ν₁₀=.11, SE=.04, df=89, p<.01). This effect remained (ν₂₀=.15, SE=.04, df=87, p<.05) when entering two covariates: weeks worked in the previous five years and negative symptoms. We also evaluated the relationship between service intensity required per week of work and found similar results; therefore, the results are not reported.

The interactions between individual characteristics and IPS contacts on weeks worked were also explored (Tables 3,4). One interaction of interest was that of mental health service hours. When IPS contacts and mental health hours were entered into the model simultaneously, IPS contacts remained significantly associated with weeks worked, whereas mental health hours did not. There was no significant interaction effect. When entered into the same model, IPS contacts were associated with weeks worked (β=.40). The interaction between cognitive symptoms and vocational contacts was significant; the relationship between IPS contacts and weeks worked decreased as cognitive symptoms increased (Table 3).

This study supported the relationship between service intensity and vocational outcomes. An important question for service providers is, “What intensity is related to continued employment?” In this sample, IPS clients actively engaged in services received slightly more than two IPS contacts per month. This rate is lower than might be expected of an “intensive” service model and is substantially lower than that reported in earlier studies, including a major multisite study (14). However, the intensity is comparable to that found by Bybee and colleagues (10) and Bond and Kukla (12). The rate in the study presented here is restricted to services documented in a Thresholds billing database and excludes vocational service contacts outside of the IPS model (for example, activities without the client present). Services include assistance both on and off the job, but they must include face-to-face contact with the consumer.

The study presented here further supported a connection between service intensity and vocational outcomes (8,13,14,24). This study suggests a possible “time-lag effect” in which the effect of employment services may not be realized until several months later. These time-lag findings suggest a temporal precedence of IPS service and are consistent with IPS services requiring intensive front-end job development and initial job placement, followed by a tapering off of employment specialist contact. The identification of factors related to vocational success within IPS is of particular importance because participants receiving IPS services still worked only a portion of the total study period.

In general, client background characteristics did not predict service intensity. However, one relationship indicated that participants who were higher functioning (that is, fewer previous hospitalizations) received more services. In contrast, McGurk and colleagues (16,25) found that having more severe symptoms was associated with more intense services per hour of work. It may be that recent hospitalizations affect services differently than symptoms; moreover, McGurk examined services to currently employed people whereas the study presented here examined both employed and unemployed participants.

Two types of symptoms moderated the relationship between service intensity and weeks worked. Participants with more severe hostility or cognitive symptoms did not benefit from increased service intensity to the extent that people with less severe symptoms did. One possibility is that people with more severe symptoms may not be able to absorb feedback at the rate that other people can. Conversely, the results of the time-lag analysis suggest the importance of job placement services for participants. However, participants with more severe cognitive symptoms may benefit more from follow-along services, which may be deemphasized in routine practice. Another confounding factor that was not explored is overall model fidelity—these results were in the context of a high-fidelity supported employment program and may not generalize to low-fidelity sites.

It is important to note that mental health services were not directly associated with weeks worked and did not moderate the effect of IPS services on weeks worked. This result is particularly notable for two reasons. First, only one other study (14) has explored the additive effects of mental health and vocational services on weeks worked. Second, mental health services were approximately nine times as intense as vocational services (assuming an average of one hour per vocational contact). Although these results argue against the additive effects of mental health services in promoting vocational outcomes, one should note the variability in types of mental health service hours and that the IPS model prescribes the integration of IPS and mental health services (26). Moreover, it may be that adequate mental health services control the effects of mental health symptoms and therefore mask the effects of both mental health services and symptoms on employment.

A substantial proportion of participants had no record of IPS contacts in some quarters; these observations were treated as zero contacts, but some may have been instances of underreporting. Missing data are attributable to two main sources: either employment specialists provided services but failed to document services in the database or participants did not receive services. Much of the missing data is likely a result of underreporting by employment specialists, who were initially unfamiliar with the database and billing codes. Also, we know anecdotally that clients sometime pause in the vocational process, resulting in service gaps. Regardless of the reasons, the study's missing data present a substantial threat to internal validity. Finally, billing databases do not account for nonbillable services—that is, times during which the service recipient was not present. Therefore, services such as calling potential employers and conferring with service providers or job supervisors were systematically excluded. Because of the missing data and nonbillable services, the average service intensity documented in this study should be considered an underestimation of actual services in an IPS program.

The large proportion of program dropouts is another limitation. These dropouts likely differed in unknown ways from program continuers. This systematic difference would again bias the sample; most likely this bias would eliminate the less motivated or work-ready participants, thus leaving a more successful sample. However, most participants received services for at least one year, and attrition was lower for IPS services than for the comparison program (27). Furthermore, it should be noted that these data were collected several years ago and current employment data may differ.

The study presented here tested several study hypotheses, as well as numerous exploratory analyses, there by increasing the threat of alpha inflation. Results from exploratory analyses should be viewed with particular skepticism given the lack of a priori theoretical support. Finally, cognitive functioning was measured by a symptom measure rather than by more thorough neurocognitive measures.

More effective methods for collecting service data are essential for the further study of psychiatric rehabilitation services. One promising approach is the use of service logs paired with regular reminders to employment specialists. Also, new technologies would make the recording of service data less time-intensive for employment specialists and more timely and accurate (28). Future research should also focus on the interplay between the service recipient and the various factors involved in his or her employment search, such as the relationship between the employment specialist and the client.