Fidelity of Supported Employment Programs and Employment Outcomes

The sample for this study comprised 2,639 individuals aged 18 to 64 years who had been diagnosed as having severe and persistent mental illness and who were enrolled in community rehabilitation and treatment programs at ten mental health centers in Vermont between October 1 and December 31, 1999. The largest of the programs had 533 participants, and the smallest had 91.

The case manager or another staff member completed an employment survey for each eligible client that included questions about competitive employment, average number of hours worked, and number of weeks worked in the last quarter of 1999. This data collection method has been validated by the Mental Health Statistics Improvement Project of the Center for Mental Health Services.

Researchers also visited the ten mental health centers to learn how employment services were provided. For each center, the researchers completed the Individual Placement and Support Fidelity Scale (4). This 15-item scale measures components of supported employment in three categories: staffing, organization, and services. Each component is rated on a 5-point anchored dimension. A score of 1 signifies "not implemented," and a score of 5 signifies "fully implemented"; thus, the lowest possible score is 15, and 75 is a perfect score. The scale has good interrater reliability and internal consistency, and it clearly differentiates supported employment programs from other types of vocational rehabilitation programs (4). We used Pearson's correlation to compare fidelity scores with employment rates.

The quarterly rate of competitive employment among the ten community mental health centers ranged from 14.4 percent to 26.4 percent (mean±SD= 21.5±3.6 percent). The total scores on the Individual Placement and Support Fidelity Scale ranged from 45 to 68 (mean=61.3± 7). The mean total score on the scale was significantly correlated with the mean competitive employment level (r=.76; p≤.01).

Sufficient variance in scores was found for seven of the 15 program components to warrant further examination. However, because of the small number of programs in the sample and the inherently lower reliability of individual component scores, the analyses for these components must be considered exploratory. As shown in Table 1, significant correlations with competitive employment were found for two components: community-based services and full-time employment specialists. Moderate correlations were found for most of the other components, although these correlations were not statistically significant.

This study provides preliminary validation of an overall supported employment fidelity scale. Several aspects of supported employment have been validated individually in previous studies, but this is the first study that we know of that shows evidence for a supported employment scale as a whole.

Despite its small sample size, the study has several strengths. It was conducted in a routine mental health system with naturalistic variation in a state system that is beginning to emphasize employment outcomes. The mental health centers were comparable in terms of funding, regulations, and state contracts, but their supported employment programs varied in several respects. Additionally, the methods of assessing employment outcomes and supported employment components have been validated in previous research.

This study points out the importance of several aspects of the supported employment approach. The two individual components that were the most strongly correlated with better employment outcomes were provision of community-based services and use of employment specialists. Provision of community-based services means that engagement, job finding, and follow-along supports are provided in community settings, away from the mental health center. The advantage of the community-based approach is that it reduces the need for people to transfer skills from one setting to another; it facilitates a more naturalistic identification of clients' interests, strengths, and natural supports, which may not have been clear in an office setting; and it builds relationships with employers directly, thereby increasing the effectiveness of job searches.

The correlation between the level of competitive employment and the presence of employment specialists reflects the importance of having staff members who provide only vocational services. Vocational workers who are assigned case management or day program responsibilities are distracted from the employment goal when they must handle other pressing issues, such as crises, illness exacerbations, housing problems, or other emergencies.

Other program components with strong or moderate correlations with competitive employment included a zero-exclusion policy, team-based integration, a vocational unit, and work-based assessment. The zero-exclusion policy means that individuals are not screened for work readiness; everyone is encouraged to consider work opportunities and is supported in these efforts. Team-based integration indicates that the employment specialists are part of the treatment team, so that mental health treatment and rehabilitation are coordinated.

Vocational unit refers to how employment specialists form a group for back-up and support. During weekly supervised group meetings, they discuss clients and share job development leads. Work-based assessment reflects a deemphasis of prevocational assessment; instead, vocational assessment is based on work experiences in competitive jobs. Additional components may also be critical, but they could not be examined here because of lack of variance.

This pilot study suggests that the supported employment components it measured account for about 50 percent of the variance in competitive employment outcomes. Other factors, such as the relationship with local offices of the Vermont Division of Vocational Rehabilitation (7), the proportion of the budget devoted to supported employment services (9), local employment conditions, staffing capacity of employment specialists, and demographic characteristics, could account for additional variance.

Most of the programs scored in the midrange of the supported employment scale, indicating that there is clearly room for improvement. Studies to determine whether improvements result in better employment outcomes are planned.

This study provides preliminary evidence that faithful implementation of key aspects of a supported employment program is associated with competitive employment outcomes. Providing supported employment services in the community and using full-time employment specialists appear to be critical components of such programs. We recommend that this study be replicated in other systems and in longitudinal studies to confirm our findings.

Work on this paper was supported by a grant from the Social Security Administration and the Substance Abuse and Mental Health Services Administration and grant MH-56147 from the National Institute of Mental Health. The authors thank Laura Flint and Boyd Tracy, M.Ed., for their contributions.

Ms. Becker and Dr. Drake are with the New Hampshire-Dartmouth Psychiatric Research Center, 2 Whipple Place, Lebanon, New Hampshire 03766 (e-mail, [email protected]). Mr. Smith and Mr. Tremblay are affiliated with the Vermont Division of Vocational Rehabilitation in Waterbury. Ms. Tanzman is affiliated with the Vermont Department of Developmental and Mental Health Services in Waterbury.