Integration of Psychiatric and Vocational Services: A Multisite Randomized, Controlled Trial of Supported Employment

The Employment Intervention Demonstration Program was funded in 1995, with the selection of eight study sites located in Maryland, Connecticut, South Carolina, Pennsylvania, Arizona, Massachusetts, Maine, and Texas. Organized under the cooperative agreement funding mechanism (Request for Applications SM 94-09), researchers and federal personnel collaborated in the development and implementation of a common protocol of research instruments, uniform data collection methods, and statistical analyses (15). This effort was led by a coordinating center based at the Department of Psychiatry at the University of Illinois at Chicago in partnership with the Human Services Research Institute located in Cambridge, Mass.

All of the study subjects met diagnosis, duration, and disability criteria for severe and persistent mental illness, as defined by the federal Center for Mental Health Services (15), along with the following inclusion criteria: age 18 years or older, interest in working, and ability to provide informed consent. At all sites, the subjects were recruited from clinical populations by provider referral, self-referral, family referral, or word of mouth. Advertisements in newspapers were also used by the Massachusetts site. Each site received approval from its local internal review board for the protection of human subjects and obtained written informed consent from all participants. The eligible pool of study participants numbered 10,653; of this group, 2,883 were contacted about participation (the numbers excluded the Massachusetts site, which was unable to provide this information). Across all sites (including Massachusetts), 1,750 individuals consented to participate in the study, 1,655 completed the first interview, and 1,648 were randomly assigned. Among those who agreed to participate but were never randomly assigned, the reasons included patient refusal, patient noneligibility, and patient loss to follow-up.

Of the 1,648 who were randomly assigned to groups, 375 were excluded from this analysis for the following reasons:

No participants were excluded for any other reason, given the “intent to treat” design of the demonstration. Recruitment occurred between February 1996 and May 2000, and all participants received a monetary stipend for each interview. A total of 1,273 subjects were included in the analysis, distributed as follows: Maine (N=108), Connecticut (N=133), Massachusetts (N=166), Maryland (N=197), South Carolina (N=142), Texas (N=233), and Arizona (N=294).

Given the study’s design as a randomized implementation effectiveness trial (5), the sites tested different models of supported employment and compared them to a variety of conditions. At each site, the experimental condition was a form of enhanced best-practice supported employment (13) compared to either services as usual or an unenhanced version of the experimental model. The Maryland, Connecticut, and South Carolina sites tested the individual placement and support model in which multidisciplinary provider teams engage in minimal prevocational assessment, rapid job search, and placement into competitive jobs, with the provision of training and ongoing follow-along support for as long as the patient requests it. The Massachusetts site used the program of assertive community treatment vocational model, with services provided exclusively in the community through a mobile team comprised of psychiatrists, nurses, case managers, and vocational specialists who collaborate in placing patients in competitive employment and providing job training and continuous employment support. The remaining three sites used models developed especially for the Employment Intervention Demonstration Program. The Texas site combined “rapid placement” supported employment services with social network enhancements that were specifically designed to help patients create more balanced and reciprocal interpersonal networks that supported their work efforts. The Maine site used family-aided assertive community treatment teams whose vocational staff worked with an employer consortium of the area’s major businesses to develop job opportunities, workplace supports, and reasonable accommodations for workers with severe mental illness. The Arizona site used an integrated treatment team composed of psychiatrists, case managers, rehabilitation counselors, employment specialists, job developers, and benefits specialists emphasizing rapid job placement and ongoing support for job retention and career advancement.

Because of ethical considerations, none of the sites used a “no treatment” comparison condition. Since the subjects entered the study desiring vocational rehabilitation, it was neither possible nor desirable to prevent them from receiving employment services. Thus, as with many multisite studies, the nature of the comparison conditions varied. Arizona, Connecticut, Maryland, and South Carolina used a services-as-usual comparison condition in which the subjects received whatever services were available in the local community. Massachusetts used the Clubhouse model (16, 17), in which facility-based services were provided according to a work-ordered day, with patients and staff working together on jobs within the program as well as at job placements in the community. Both Texas and Maine used an “unenhanced” version of their experimental condition (i.e., no social network services in Texas and no employer consortium in Maine).

All of the experimental programs integrated the provision of vocational and psychiatric services through multidisciplinary teams. Team members adhered to a program philosophy emphasizing rapid, permanent, competitive employment in a field of the patient’s choosing, followed by ongoing support with no time limits. However, at some sites, the comparison model also involved delivery of highly integrated psychiatric and vocational services. Therefore, to operationalize the level of integration, an Employment Intervention Demonstration Program measure was developed in which programs were rated independently according to the following criteria:

“Multidisciplinary teams” were defined as designated units that included (at a minimum) psychiatrists, case managers, job developers, and employment support staff who met in person usually daily but no less than 3 times per week. “Same location” was defined as having offices in a single building and “same agency” as a single organizational unit. “Single case record” was defined as a file that incorporated employment assessments and treatment plans, vocational outcome data, medication information, and case management notes. Each of the 14 programs was rated on each of the four integration criteria: four programs met 100% of the criteria, three met 75%, two met 50%, one met 25%, and four met 0%, resulting in a service-delivery integration variable with a mean of 53 and a median of 62. Inspection of this score revealed that its distribution was bimodal; therefore, the services-integration score was dichotomized so that programs meeting less than 50% of the criteria were classified as low in integration and otherwise as high.

Dichotomous variables based on the subjects’ reports at baseline were used for gender, minority status (white/Caucasian versus all others), and education (high school or more). Age was measured in 10-year intervals. The Structured Clinical Interview for DSM-IV (18) was administered at two sites by the project researchers, while case record diagnoses made by treating psychiatrists were extracted from clinical files at the remaining sites. Level of functioning, defined as the ability to function in non-work-related domains, such as independent living and social relationships, was self-rated by the subjects using a single Likert-scale item with response options of “poor,” “fair,” “good,” or “excellent.” Prior work history was reported by the subjects and coded as one or more jobs in the past 5 years versus none. Receipt of public disability income was reported by the subjects and coded as Supplemental Security Income (SSI) alone, Social Security Disability Income (SSDI) alone, or SSI plus SSDI (with no public disability income as a contrast). An independent psychometric evaluation of the Employment Intervention Demonstration Program’s common protocol (19) found good to excellent validity and reliability on the measures described.

Of the 1,273 participants, 824 (65%) completed five interviews, 173 (14%) completed four interviews, 122 (9%) completed three, 111 (9%) completed two, and the remaining 43 (3%) completed one interview. Those completing five interviews were compared to all others regarding model covariates. The only significant differences were in gender and age: 51% (N=420) of completers were men versus 58% (N=258) of the noncompleters (χ²=4.92, df=1, p<0.05); the completers were 1 year older than the noncompleters (mean=39, SD=9, versus mean=38, SD=10) (t=–2.24, df=1270, p<0.05).

After calculation and inspection of frequency distributions and zero-order relationships, outcomes were inspected visually for each of the 24 months of study participation. Next, random-effects logistic regression modeling, part of a family of statistical methods termed random regression models, was used to address hypotheses at the multivariate level. This approach was chosen to address issues commonly found in longitudinal multisite data, including serial correlation, individual heterogeneity, missing observations, and fixed versus time-varying covariates (20). All models included patient demographic characteristics (age, gender, race/ethnicity, education, work history, and SSI/SSDI status), clinical factors (schizophrenia spectrum diagnosis, level of functioning), time, and study site. In step 1, the effects of high versus low service integration were entered into the model; in step 2, the effects of the vocational and psychiatric services measures were added to the previous model, including the services-integration variable. Given the hypothesis-driven nature of the study, higher-order interactions between model variables were assumed to be nonsignificant.

Just over half of the subjects (N=678, 53%) were men, two-thirds (N=839, 66%) had a high school education or greater, half (N=636, 50%) were Caucasian, 30% (N=380) were African American, 14% (N=184) were Hispanic/Latino, 3% were American Indian, and the remainder were of other races/ethnicities. Their ages ranged from 18 to 76 years, with a mean and median of 38. The majority (N=763, 64%) reported holding one or more jobs in the 5 years before the baseline. Self-rated levels of functioning at baseline were characterized as poor by 15% (N=189), fair by 40% (N=509), good by 31% (N=396), and excellent by 14% (N=172). At baseline, 35% (N=437) reported receiving SSI, 25% (N=307) received SSDI, 12% (N=153) received both, and 28% (N=353) received neither. On axis I, half of the subjects (N=646, 51%) had a primary or secondary diagnosis of schizophrenia spectrum disorder, 21% (N=264) had major depression, 16% (N=206) had any bipolar disorder, 4% (N=45) had dysthymia or depressive disorder, 3% (N=43) had any psychosis, 1% (N=13) had posttraumatic stress disorder, and the remaining 4% (N=56) had other disorders, including anxiety, panic, delusional, dissociative, obsessive-compulsive, and somatization disorders.

Although all experimental condition programs met the criterion for high integration, at two of the seven sites (Maine and Texas), the comparison conditions also met this criterion. Thus, more study participants were served by programs with high levels of integration and fewer by programs with low levels of integration (N=811, 64%, versus N=462, 36%, respectively). Table 1 presents analyses of subject characteristics by high versus low level of service-delivery integration. There were no differences between the two groups in education, gender, diagnosis, level of functioning, age, or SSDI and SSI/SSDI beneficiary status. There were significant differences: a higher proportion of Caucasians as well as patients with recent work histories and a lower proportion of SSI beneficiaries were enrolled in high-integration programs. These three factors were controlled in all subsequent phases of the analysis. In addition, the study site was also controlled to account for the classification of all Maine and Texas subjects as receiving highly integrated services, as well as other potential differences among sites that may have remained unmeasured and unaccounted for. Even though integration was operationalized separately from hours of vocational and clinical services, there was a weak yet significant correlation between total average hours of vocational services and integration (r=0.19, p<0.0001), in which patients in high-integration programs received a greater number of hours of vocational services than those in low-integration programs. However, there was no difference in psychiatric services between the high- and low-integration programs.

Figure 1 shows the average (per person) number of hours of psychiatric and vocational services received by all patients for each of their 24 months of program participation, with adjustment for the subjects’ varying calendar months of participation. Starting with their first month, on average, the patients received twice as many hours of psychiatric services as vocational services: an average of 7.2 hours of psychiatric and 1.9 hours of vocational services in month 1, 4.7 hours versus 2.6 hours in month 12, and 4.2 hours versus 1.7 in month 24). In addition, the average monthly hours of service per patient decreased over time for both psychiatric and vocational services.

As reported in a separately published analysis of the original experimental versus comparison study conditions (21), there were significant differences by study condition for both vocational outcomes. Over the course of the study, 55% (N=359) of the experimental participants achieved competitive employment compared to 34% (N=210) of the subjects in the comparison condition (p<0.001). A similar but more modest difference was seen for the second vocational outcome, with 51% (N=330) of the subjects in the experimental condition having worked for at least 40 hours in at least 1 month compared to 39% (N=245) of the subjects in the comparison condition (p<0.001).

To test the present study’s first hypothesis, client outcomes were compared according to whether they were served in models with high versus low integration of psychiatric and vocational service delivery. Over the 24-month course of the study, a larger proportion (N=471, 58%) of the participants in the high-integration services programs achieved competitive employment compared to 21% (N=98) of the subjects in the low-integration programs (χ²=161.8, df=1, p<0.001). For the second vocational outcome, working 40 or more hours in a month, more than half (N=431, 53%) of the high-integration participants worked for at least 40 hours in a month compared to 31% (N=144) of the low-integration participants (χ²=57.4, df=1, p<0.001).

Figure 2 and Figure 3 show the proportion of patients over time who achieved an outcome by their month of participation in the study. Starting with month 1 and continuing throughout the study, a larger proportion of the high-integration participants achieved competitive employment (Figure 2) and 40 or more hours of work in a month (Figure 3) than did the low-integration participants. The highest proportion working competitively in any month of study participation was around 27%, achieved by high-integration patients, whereas the highest percentage working 40 or more hours in a month was around 25%, again achieved by the high-integration group. These findings are consistent with hypothesis 1.

To test hypothesis 1 at the multivariate level, random regression models were computed by using high versus low service-delivery integration as the independent variable. In step 1, patient demographic and clinical characteristics, time, and study site were entered along with the level of integration. In step 2, the effects of hours of vocational and psychiatric services were added to the model used in step 1. As shown in Table 2, the estimated odds ratios for the two outcome variables were significant, so that participants in programs with high integration had better outcomes than those with low integration. Specifically, participants in high-integration programs were more than twice as likely to work competitively and were 1¼ times as likely to work 40 or more hours in a month. This analysis also controlled for study site as well as client demographic, illness, work history, and disability income beneficiary features.

In step 2, the models were further specified by the addition of two covariates: one measuring monthly cumulative hours of psychiatric services and a separate cumulative measure of monthly vocational services. The results in step 2 (Table 2) show that the total cumulative hours of vocational services were significantly associated with a greater likelihood of both competitive employment and working ≥40 hours per month. This is indicated by the fact that the odds ratio for vocational services measured in hours exceeded 1. On the other hand, the total cumulative hours of psychiatric services were significantly associated with the lesser likelihood of the two outcomes. This is shown by the fact that the odds ratio for hours of psychiatric services was lower than 1.

The results of step 2 also show that even after we controlled for the cumulative amounts of both types of services, patients in highly integrated service-delivery programs had superior employment outcomes. This is indicated by the fact that the two odds ratios for the integration effect remained significant for both outcomes in step 2 and also increased in size. Thus, service-delivery integration had a distinct and significant association with work outcome.

Because the two services variables were entered into the models as interval-level variables, their odds ratios are somewhat difficult to interpret. To clarify the association between services and outcomes, we dichotomized the two service variables by dividing them at the median for each month, classifying respondents as receiving “high” versus “low” amounts (in hours) of each. This allowed us to examine the effects of services when they were operationalized in a manner similar to that used for the other model variables. After we entered these dichotomized services variables in step 2 of the models described earlier, the participants receiving a high number of hours of vocational services were almost 2½ times as likely to work competitively (odds ratio=2.41, p<0.001) and were almost twice as likely to work 40 or more hours in a month (odds ratio=1.97, p<0.001). Conversely, psychiatric services had a weaker effect on vocational outcomes: those receiving high amounts of psychiatric services were four-fifths as likely to work competitively (odds ratio=0.79, p<0.001) and were 7% less likely to work ≥40 hours per month (odds ratio=0.93, p<0.001).

A final step in our analysis involved testing for an interaction effect between the two types of services. When we used step 2 variables in our models, the results (not shown) revealed a significant interaction effect in which individuals who received high amounts of vocational services did best, regardless of whether they also received high or low amounts of psychiatric services. Thus, the intensity of employment services was related to success, regardless of the intensity of the psychiatric services received.