Outcomes of Peer-Provided Individual Placement and Support Services in a Mental Health Peer-Run Vocational Program

This study took place at Baltic Street Advocacy, Employment, and Housing, Inc. in New York City from July 2015 to July 2017. The agency provided advocacy and self-help services, support groups, residential assistance, and services bridging the transition from hospital to community. The agency’s intake process required referral from a treating clinician who completed a psychosocial assessment before admission confirming serious mental illness. The program had delivered vocational services for several years, loosely based on the choose-get-keep supported employment model (31, 32). Two vocational teams using identical staffing (a team leader plus two to three full-time-equivalent staff members), supervision format, and job descriptions were located at separate offices in different areas of the same city borough with similar geographic and job market features. Staff on each team were completely nonoverlapping.

Study participants included all supported employment recipients during the study period. Individuals enrolled in employment services by expressing interest in working or being referred from the larger program or external programs. Assignment to one or the other team was based on participants’ personal preference and convenience. Their deidentified vocational outcome data were used, regardless of participation in employment services. Another data source was information from IPS fidelity reviews of the intervention condition conducted at study start, midpoint, and end. The study was approved by the institutional review board of the University of Illinois Chicago.

Because random assignment of participants was impractical, we used a quasi-experimental design. The intervention consisted of IPS services combined with health promotion services delivered by employment specialists. IPS services involved completion of a vocational profile summarizing the person’s employment resources and strengths, work history, and desired type of job and working conditions. Employment specialists provided individualized, intensive job search support, including liaison with potential employers and coaching during the interview and hiring process. After employment was secured, ongoing support was provided to the worker and the employer (if desired) with no time limits. The physical wellness component of the intervention involved a structured set of activities during meetings with employment specialists. These activities included education about work-health connections regarding sleep and rest, physical activity, relaxation exercises and stress management, healthy eating, and medical care. Participants used the Physical Wellness for Work planning tool (https://www.center4healthandsdc.org/physical-wellness-for-work.html) to identify health habits and routines to support their specific work goals. For each goal, participants identified steps to take to establish health routines, find resources required, anticipate obstacles, seek assistance needed from employment specialists, and initiate accountability methods such as check-ins or reminder calls. During subsequent meetings, employment specialists inquired about progress toward establishing the desired health routines, helped remove any barriers, and modeled their own work-related wellness strategies. Specific activity codes were used to document in the person’s case file each step of this process (didactic instruction, planning tool completion, and identification of work-related habits and routines) and degree of progress made in changing health routines.

The comparison condition was usual vocational services delivered by peer staff on the second team, using the choose-get-keep approach (32). Services included supporting participants who engaged in preemployment activities to identify their preferred jobs and work settings. This step was followed by assistance during the hiring and onboarding process and then by support for sustaining employment through training and follow-along as well as help during job exit (if needed) and afterward (31).

The team that delivered IPS received training and supervision throughout the study from two external experts who were affiliated with the IPS Employment Center, the model’s national training and research entity. The IPS training expert (a private consultant) familiarized staff with IPS principles and practices, modeled and observed IPS service delivery activities, and provided ongoing supervision and support for employer outreach, job development, and support activities. Initial training involved assigned readings, completing the IPS Employment Center’s practitioner skills course, expert feedback at twice-monthly onsite visits, and weekly telephone supervision. Training was repeated whenever new staff were hired. Performance goals were established for employment specialists and included monthly number of new employer contacts and new jobs developed. Periodic IPS fidelity assessments identified areas for further training.

External health and wellness experts from peer-run Collaborative Support Programs of New Jersey also worked with IPS staff throughout the study period. They introduced tools and activities to help participants address general health issues that affect employment. This training included assigned readings, didactic instruction on links between physical wellness and successful job search and retention, wellness goal setting with specific wellness tools, modeling wellness tools for participants, and ongoing technical assistance via in-person visits and conference calls for mentoring and supervision. Additionally, the experts worked with managers to create logs and a notation system for documenting health promotion services and progress in the participant’s file.

The primary outcome was competitive employment defined in part 363 of the Rehabilitation Act of 1973 as amended (33) as a job in the competitive labor market belonging to the worker and not representing a set-aside for people with disabilities, paying minimum wage or above, and located in a socially integrated setting (34). Researchers verified that jobs met the definition of competitive employment. Monthly data on competitive employment status included hours worked, hourly wage, and job start and end dates. Participant background data included age, gender, race, ethnicity, education, number of months receiving employment services, and program year. Program measures for each team included caseload size, number of job starts, and job tenure.

Recognized as a critical component of the IPS model, the Supported Employment Fidelity Scale (35) was used to assess fidelity before the start of the study, midway through the study, and after study completion. Possible scores on this scale range from 25 to 125, with ≤73 indicating lack of supported employment; 74–99, fair fidelity; 100–114, good fidelity; and 115–125, exemplary fidelity. Following standard assessment procedures (36), an external expert and a trained partner conducted interviews with the agency chief executive officer, vocational program director, IPS team leader, IPS employment specialists, and a sample of IPS service recipients. Ten randomly selected client files also were reviewed, along with vocational outcome data. This information was then used to complete scale items divided into three sections assessing staffing, organization, and services.

We assessed statistical significance of differences between teams with chi-square and t tests. We modeled the primary outcome of competitive employment by using a random-effects logistic regression model (RRM) with group assignment propensity score adjustment. The data’s longitudinal, repeated-measures structure, with varying number of measurements per participant, was appropriately handled by RRM (37). Given the quasi-experimental design, propensity score adjustment compensated for nonrandomized study group assignment (38, 39). We estimated propensity scores in a logistic regression model predicting team assignment (IPS vs. comparison team) by using six participant characteristics: age, gender, race, ethnicity, education, and number of months receiving vocational services. The RRM had an autoregressive covariance structure and included study condition (IPS vs. comparison condition) and calendar time (months 1–25). The model also controlled for monthly caseload of teams and number of months services were received. We modeled differences in the rate of change in competitive employment over time by using a condition × time interaction term. Analyses were conducted in IBM SPSS Statistics, version 25.0.

Programs are considered to be implementing IPS if they reach the threshold of fair fidelity (40), and studies of multisite IPS implementation consider model adoption to be successful when programs reach “fair to good” fidelity (41). At study baseline, the intervention team received a total fidelity score of 71, indicating that its services did not qualify as IPS supported employment. Feedback from consultants included advice to streamline the intake process, emphasize the employment specialists’ leadership role in the job search, and secure competitive jobs in a wider range of community settings. At study midpoint, a score of 99 was achieved, indicating “fair fidelity.” Consultant feedback included providing skill enhancement across different IPS service phases, creating an IPS procedural manual, and adopting monthly performance benchmarks for employment specialists. At the third and final fidelity assessment, a total score of 110 was achieved, indicating good fidelity. Consultants’ observations included kudos for adding a benefits counselor to the team, compliments for becoming a Ticket to Work provider and thus broadening sources of program revenue, suggestions for increasing participants’ employment rates, and acknowledgment of the team’s recent Recognition of Excellence for Wellness Award from the Substance Abuse and Mental Health Services Administration for its innovative blend of health, wellness, and IPS employment services.

Table 1 presents the characteristics of 348 individuals served by the two teams (IPS, N=184; comparison condition, N=164) over the 2-year period, along with team-level features, such as average caseload and number of clients served. The teams did not differ statistically significantly in any participant or team characteristics except for race, with the IPS team having a higher proportion of Black participants and a lower proportion of White participants than the comparison team.

Employment outcomes are shown in Table 2. Overall, 43% of IPS participants achieved competitive employment compared with 21% of comparison participants (p<0.001). On average, 38% of IPS participants were in competitive employment each month compared with 18% of comparison participants (p<0.001). The IPS group had a significantly higher mean±SD number of job starts per month than the comparison group (3.4±2.7 vs. 0.9±0.9, p<0.001). Among those with jobs that ended during the study period (N=39), the IPS group had a longer mean job tenure than the comparison group: 134.1±133.6 days for the IPS group compared with 75.2±31.5 days for the comparison group (p=0.033). Finally, the groups did not differ in mean hourly wage ($13.25±$5.54 per hour) or mean hours worked per week in competitive employment (26.0±13.2).

Rates of competitive employment per month for each group are shown in Figure 1. In the first month of the observation period (July 2015), the IPS group’s rate was 14% (N=6 of 42) compared with 22% (N=10 of 46) for the comparison group. The IPS group’s rate increased across 25 months of observation to end at a high of 73% (N=51 of 70) in July 2017. Over the same period, the comparison group’s rate remained about the same, with declines in some months, ending at a low of 9% (N=6 of 69).

The results of the longitudinal RRM adjusted for propensity scores are shown in Table 3. Across the entire study period, participants in the IPS group were more than four times as likely to achieve competitive employment as participants in the comparison group. We noted a modest increase in the likelihood of competitive employment over time for both groups. Service month also was statistically significant, indicating that each additional month of employment services was associated with a substantial increase in the likelihood of obtaining competitive employment. Finally, caseload size was not associated with a change in likelihood of competitive employment.

Table 4 shows the rate of change in competitive employment over time, including the interaction of team assignment and time. At baseline, the IPS group had a significantly lower rate of competitive employment than the comparison group, and the rate of competitive employment in the comparison group declined over time. We observed a significant and positive interaction of IPS and time, indicating that, relative to the comparison group, the rate of competitive employment in the IPS group increased significantly over time. By multiplying the relative increase in the IPS group by the rate of competitive employment in the comparison group (1.32×0.91=1.20), we estimate that the odds of competitive employment in the IPS group increased by about 20% per month during the study period. Once again, participant service month was significantly associated with achieving competitive employment, whereas team caseload size was not.