Contents and Intensity of Services in Low- and High-Fidelity Programs for Supported Employment: Results of a Longitudinal Survey
Abstract
Objective:
Little is known about the association between service intensity and fidelity scale score in supported employment programs. This study compares service contents and intensity in low- and high-fidelity programs and examines the validity of the Japanese version of the individualized Supported Employment Fidelity Scale.
Methods:
The vocational outcomes and service provision data for 51 individuals with schizophrenia in 13 supported employment programs were collected over a 12-month study period. Outcomes, service contents, and service intensity were compared between the low-fidelity group (seven programs; N=29) and the high-fidelity group (six programs; N=22).
Results:
In both groups, 70% of the total services (hours) were provided in the first 6 months. The high-fidelity group, which was associated with better vocational outcomes than the low-fidelity group (employment rate, 68% versus 38%, respectively), made the greatest effort in job development outside of the agency, whereas the low-fidelity group spent more time on group services. In addition, before the client obtained a job, high-fidelity programs provided outreach services (B=7.2, p=0.043) and agency-based individual services (B=5.7, p<0.001) at greater intensity than did low-fidelity programs. However, no significant between-group difference was found in service intensity once clients were employed.
Conclusions:
Supported employment programs with a high fidelity score focus more intensely on providing individual services in and outside of the agency, particularly before clients obtain a job. However, clarification of the relationships among service quality at the structure level, amount of follow-up services, and individual needs in supported employment programs is a future issue.
HIGHLIGHTS
•
In Japan, high-fidelity supported employment programs created better vocational outcomes.
•
High-fidelity supported employment programs focused more intensely on providing individual services in and outside of the agency before clients obtained jobs.
•
No significant difference was found in service intensity between low- and high-fidelity supported employment programs once clients were employed.
Employment for people with severe mental illness is a global issue. Previous studies of supported employment, particularly those concerning the individual placement and support (IPS) model, have developed international evidence of its effect on vocational outcomes (1–6). In this context, research interest has gradually shifted from investigating the effectiveness of supported employment to examining implementation and dissemination issues in real-world settings through evaluations of the service structure and service processes (7, 8).
In terms of evaluation of service structure, several studies have conducted fidelity scale development research to measure quality of services in each agency at the structure level (e.g., on the 25-item IPS [IPS-25]) (9–11). The Japanese version of the individualized supported employment fidelity scale (JiSEF) was also developed through modification of IPS-25 to create an evidence-based supported employment program fidelity scale that fits Japanese social and labor service systems (11). Previous fidelity development studies compared fidelity scores with vocational outcomes or self-assessment scores (9–14) but rarely focused on the association with the service process. With regard to service processes, past studies have addressed process evaluations by assessing the contents and intensity of supported employment programs, but the results have been inconsistent.
For example, U.S. studies of process evaluation on supported employment services have reported that clients contact IPS program 2–3 times per month (15) or receive vocational services for approximately 2–3 hours per month (16) and mental health services for approximately 4–19 hours per month (15, 16). A Japanese pilot study also reported that the mean service hours for vocational and mental health services in supported employment programs was approximately 11 hours (17). Although greater service intensity was likely associated with finding competitive employment, the relationship between service intensity and work tenure varied between the studies (15, 18–20). One reason for these heterogeneous findings regarding the service process may be differences among supported employment programs in quality of service structure. For example, programs with high-quality service structure, as indicated by a high score on the fidelity scale, may provide more services and, consequently, better vocational outcomes compared with those with a low fidelity score (15). However, few studies have compared the amount of services provided by programs on the basis of their fidelity to the IPS model.
Implementation and service processes also vary between countries in accordance with the relevant policies. For example, Japan has unique laws for mental health services, including employment services. These laws do not focus on individual services for people with severe mental illness. Instead, group services are offered in agency facilities regardless of disability type (6). Therefore, employment service providers often prioritize the provision of group services. Moreover, the Japanese system does not include community mental health centers that combine mental health services and employment services (6). Instead, the majority of employment services are provided by community social service agencies, which serve people with various social needs and types of disability (11). These agencies generally do not care only for clients with severe mental illness, such as schizophrenia; however, clients with a severe mental illness may exhibit severe symptoms and low cognitive function that require frequent contact with supported employment programs (21, 22). In fact, a diagnosis of schizophrenia itself has a negative effect on employment compared with other diagnoses (23–25). The unique Japanese mental health system may affect the service process, and service contents and intensity may differ from those of U.S. programs.
To address the evidence gap, we conducted fidelity research and a prospective study to assess the service provision hours and service contents of supported employment programs among clients with schizophrenia in a Japanese setting. This study aimed to examine the association between the intensity and quality of an agency’s supported employment services, as indicated by fidelity score, and to further validate the JiSEF. In particular, we investigated service contents and compared service intensity among low- and high-quality supported employment programs before clients found a job and during their employment.
Methods
Overall Design
We conducted a prospective longitudinal study with a 12-month follow-up in a routine setting. Through the scale development research for the JiSEF, we identified 25 agencies that at least partially provided supported employment programs for people with mental illness (11). Of these, 13 programs participated in the study (the flow diagram is available in Figure 1 of the online supplement to this article). Five programs were located in or near the Tokyo area, and eight programs were located in other areas. Four programs were part of medical institutions such as psychiatric day care and outpatient services, and nine were in community social service agencies. The mean±SD number of employment specialists and their caseloads were 1.9±1.6 and 25.2±24.0, respectively.
Using the JiSEF, we assessed the program fidelity of each program between August 2014 and February 2015. The JiSEF, which has 25 items (possible scoring range 25–125), was developed on the basis of the IPS-25 (10) and was modified to fit Japanese settings. For example, because the Japanese mental health care system does not have community mental health centers (6), the relevant items in JiSEF were modified from IPS-25 (see Sasaki et al. [11] and its supplementary material). The JiSEF’s convergent validity and interrater reliability have previously been confirmed (11). In addition, the JiSEF has a cutoff point (90/91), which can be used to screen supported employment programs that have high employment rates (26).
In this study, the fidelity research was conducted by researchers who received 1 day of training. We found good interrater reliability with an intraclass correlation coefficient (ICC) of 0.93. We compared the data for service provision and vocational outcomes between the low-fidelity group (mean±SD score for low-quality supported employment programs, 80.1±6.5; N=7) and the high-fidelity group (high-quality supported employment programs, 100.8±5.3; N=6).
Participants
We recruited participants at each agency between December 1, 2014, and November 30, 2015. The eligibility criteria were as follows: clients who were receiving employment services for the first time in each agency; had been diagnosed as having schizophrenia on the basis of the ICD-10; and were between ages 20 and 60 years. Written informed consent was obtained from all participants. This study was approved by the Research Ethics Committee at the National Centre of Neurology and Psychiatry (A2014–078).
Measures
Service provision data.
Because past studies have measured vocational and mental health service data separately (15, 16), the process data service inventory sheets for employment specialists and case managers used in this study were independently developed. In a Japanese pilot research project to collect process data and categorize service types, the initial inventory sheets were created in collaboration with supported employment staff members (17). Before this study was conducted, we revised the inventory sheets through an outreach service process study to clearly identify the delivery methods of the content of each service (27). The inventory sheet for employment specialists had nine codes for individual services (e.g., assessment, job counseling, job development) and one code for group services. The inventory sheet for case managers had 10 codes for individual services (e.g., assessment, individual life counseling, and individual life skill training) and one code for group services (the inventory sheet codes is available in Table 1 of the online supplement to this article).
On both inventory sheets, all individual service codes were divided into three subcodes: outreach services, agency-based individual services, and telephone and e-mail services. Outreach services means services provided outside of the agency or its offices. Agency-based individual services means individual services (rather than group services) provided by the agency staff members in facilities. The agency-based group services code was divided into four subcodes: work skills, cognitive rehabilitation, illness management, and recreation and physical activities.
During the 12-month follow-up period, employment specialists and case managers completed an inventory sheet each time they provided a participant with services. They selected the appropriate code to reflect the service provided on that day and then wrote the minutes of their service provision in each code. The study collaborator affiliated with each supported employment program regularly checked clients’ service records for omissions and incomplete data.
Vocational outcomes.
Competitive employment status data for participants during the 12-month follow-up were collected from their service records and based on their employment contract. Employment was operationally defined as working at least 1 day a month at minimum wage or more as determined by Japanese law during the follow-up period. We calculated length of time to find first job, work tenure during the study period, hours worked per week, and work income. In addition, we collected information about whether the participants maintained their employment for 1 year after starting their job, which was beyond the 12-month follow-up period. Data collection was completed in December 2017.
Background variables.
As a baseline assessment, we obtained sociodemographic variables and information from each program’s client records about participants’ employment experience for the 6 months before the study. In addition, we assessed the participants’ function using the Global Assessment of Functioning (GAF) (28). The participants also completed a scale to assess the motivation for competitive employment of people with mental illness (motivation scale) (29). The scale development study for the motivation scale confirmed its factor validity and reliability, including high internal consistency (Cronbach’s α=0.91) and high test-retest reliability (ICC=0.83) (29).
Data Analysis
We compared sample characteristics at baseline and vocational outcomes between the low- and high-fidelity groups. In terms of the service provision data, we first presented descriptive statistics for number of service uses per month during the 12-month follow-up. We calculated the number of service uses by determining the number of days on which the participants received services from either employment specialists or case managers as indicated on the inventory sheets.
Second, we computed the mean duration of services (minutes and hours per month) for each code on the basis of the inventory sheet data. The codes were classified into four categories: outreach services, agency-based services, telephone and e-mail services, and agency-based group services. We then summarized the mean hours in each category of service by month for each group. Third, we computed the mean hours per month in each category of service before the participants found a job by dividing the number of hours per service by the number of months until participants started a new job. If a participant did not find competitive employment during the follow-up period, the service hours were divided by 12 (months). Similarly, for participants who worked during the study period, we calculated the mean hours per service per month between the job start and the end of 12-month follow-up (or job termination).
On the basis of our primary interest, we implemented the generalized linear model (GLM) with the robust standard error adjusted within each program (Stata syntax, vce[cluster] option) to compare the mean hours provided in the categories of outreach services, agency-based individual services, and agency-based group services. The dependent variable was service hours, combining employment specialists and case managers, because the integration of vocational and mental health services and an interdisciplinary team approach is an essential factor of effective supported employment services (a detailed description is available in the online supplement ) (30, 31). The independent variable was group (low- or high-fidelity program). We simultaneously adjusted for employment status during the 12-month study period as a covariate when analyzing the hours of service provided before the participants found a job. We also performed a sensitivity analysis to adjust for the other potential covariates, including sex, age, educational qualifications, past work experience, GAF score, and motivation scale score (32–34). In addition, we conducted supplemental analyses to examine the association between hours of service provided and vocational outcomes using a univariate logit model and Spearman’s correlation. Statistical significance was set at the 5% level. All analyses were conducted by using Stata version 15.
Results
Sample Characteristics and Vocational Outcomes
During the recruitment period, 93 clients met the inclusion criteria, and 51 were included in the analyses (see flow diagram available in the online supplement ). No significant between-group differences were found in sample characteristics and scale scores at baseline (Table 1). As shown in Table 2, participants in the high-fidelity group were significantly more likely than those in the low-fidelity group not only to find jobs (68% versus 38%, p=0.032) but also to retain the initial job after 1 year (59.1.% versus 20.7%); they also had a longer mean work tenure (166.2 versus 56.4 days, p=0.002).
| Low-fidelity group (N=29) | High-fidelity group (N=22) | ||||||
|---|---|---|---|---|---|---|---|
| Characteristic | N | % | N | % | Test statistic | df | p |
| Sex | |||||||
| Male | 23 | 79 | 13 | 59 | χ2=2.46 | 1 | .117 |
| Age (M±SD) | 35.8±7.8 | 39.1±9.0 | t=1.39 | 49 | .171 | ||
| Education | χ2=3.10 | 4 | .542 | ||||
| Secondary | 3 | 10 | 3 | 14 | |||
| High school | 16 | 55 | 8 | 36 | |||
| Vocational school | 4 | 14 | 2 | 9 | |||
| Junior college | 1 | 4 | 2 | 9 | |||
| University | 5 | 17 | 7 | 32 | |||
| Marital status | χ2=1.95 | 2 | .377 | ||||
| Unmarried | 23 | 79 | 20 | 91 | |||
| Married | 2 | 7 | 0 | — | |||
| Divorced | 4 | 14 | 2 | 9 | |||
| Residence | χ2=.91 | 2 | .634 | ||||
| Living with family | 22 | 76 | 14 | 64 | |||
| Living alone | 6 | 21 | 7 | 32 | |||
| Residential facility | 1 | 3 | 1 | 5 | |||
| Job experience in past 6 months | 3 | 10 | 2 | 9 | Fisher’s exact test | 1.000 | |
| Global Assessment of Functioning (M±SD)a | 49.6±13.1 | 49.7±10.6 | t=.05 | 49 | .959 | ||
| Motivation scale (M±SD)b | 67.1±10.9 | 71.5±9.2 | t=1.53 | 49 | .133 | ||
a
Possible scores range from 0 to 100, with higher scores indicating better overall functioning.
b
Possible scores range from 23 to 92, with higher scores indicating greater motivation for competitive employment.
| Low-fidelity group (N=29) | High-fidelity group (N=22) | ||||||
|---|---|---|---|---|---|---|---|
| Outcome | N | % | N | % | Test statistic | df | p |
| N of people who worked | |||||||
| During the past 12 months | 11 | 37.9 | 15 | 68.2 | χ2=4.58 | 1 | .032 |
| Maintained the initial job throughout the study period (1 year after starting work) | 6 | 20.7 | 13 | 59.1 | |||
| Maintained the last job throughout the study period | 6 | 20.7 | 12 | 54.5 | |||
| Length of time to find the initial job (M±SD days) | |||||||
| All participants | 298.2±107.1 | 190.5±140.1 | t=3.12 | 49 | .003 | ||
| Participants who worked | 189.0±104.7 | 109.1±84.3 | |||||
| Work tenure (M±SD days) | |||||||
| All participants | 56.4±101.0 | 166.2±135.0 | t=3.32 | 49 | .002 | ||
| Participants who worked | 148.8±116.0 | 243.7±84.2 | |||||
| Work hours per week (M±SD)a | 22.6±12.4 | 21.1±9.6 | t=.35 | 49 | .730 | ||
| Work income (M±SD $)a,b | 4,173.9±3,874.40 | 5,080.50±4,625.80 | t=.53 | 49 | .603 | ||
a
For participants who worked.
b
Total income during the 12 months; $1 U.S.=110 Japanese yen (February 7, 2019).
Descriptive Statistics for Service Provision
We found no difference in mean service uses per month between the low-fidelity group (6.8±4.8) and the high-fidelity group (6.9±3.8) during the 12-month follow-up. However, the mean hours of total services per month were greater in the low-fidelity group (18.6±17.1) than in the high-fidelity group (10.9±7.7) (see Table 2 in the online supplement ). Overall, services provided to the high-fidelity group included a balance among outreach services (29%), agency-based individual services (36%), and agency-based group services (32%). By contrast, a large proportion of services provided to the low-fidelity group was agency-based group services (83%). For both groups, about 70% of the total hours of service were provided in the first 6 months (see Figures 2 and 3 and Table 3 in the online supplement ). With regard to the service provision codes for the employment specialists, the high-fidelity group expended more effort on outreach-based job development (136.8 minutes per month), whereas the employment specialists in the low-fidelity group spent more time on group services for work skills (349.6 minutes per month) (Table 3). For case managers in both groups, minutes of service provided were highest for recreation and physical activities (low-fidelity group, 127.7 minutes per month; high-fidelity group, 95.3 minutes per month). The case managers in the high-fidelity group also provided more individual employment services than did those in the low-fidelity group (Table 4).
| Low-fidelity group (N=29) | High-fidelity group (N=22) | |||
|---|---|---|---|---|
| Type of service | M | SD | M | SD |
| Assessment | ||||
| Outreach services | .0 | .0 | .6 | 2.9 |
| Agency-based services | 1.9 | 3.0 | 6.1 | 8.4 |
| Telephone and e-mail services | .0 | .0 | .1 | .2 |
| Job counseling | ||||
| Outreach services | 2.0 | 4.1 | .6 | 1.3 |
| Agency-based services | 37.1 | 34.3 | 42.7 | 42.2 |
| Telephone and e-mail services | 2.0 | 3.2 | 3.8 | 5.8 |
| Individual training for business skills | ||||
| Outreach services | .0 | .0 | 5.9 | 17.2 |
| Agency-based services | 10.5 | 22.9 | 31.2 | 50.2 |
| Telephone and e-mail services | .0 | .2 | .1 | .3 |
| Job development | ||||
| Outreach services | 96.1 | 148.2 | 136.8 | 87.6 |
| Agency-based services | 3.8 | 9.0 | 2.9 | 4.7 |
| Telephone and e-mail services | 1.1 | 1.9 | 5.0 | 8.9 |
| Counseling to keep job/ on-site job support | ||||
| Outreach services | 4.0 | 14.1 | 16.6 | 20.3 |
| Agency-based services | 2.7 | 6.2 | 14.2 | 23.7 |
| Telephone and e-mail services | .7 | 1.9 | 4.7 | 9.3 |
| Managing services from other agencies | ||||
| Outreach services | 1.4 | 3.1 | 8.5 | 15.9 |
| Agency-based services | 1.1 | 2.9 | 1.3 | 2.3 |
| Telephone and e-mail services | .6 | 1.4 | 1.6 | 2.1 |
| Managing services from other professionals in the same agencies | ||||
| Outreach services | .1 | .3 | .0 | .0 |
| Agency-based services | .7 | 2.0 | 1.5 | 5.3 |
| Telephone and e-mail services | .1 | .4 | .1 | .4 |
| Medical and living support from employment specialists | ||||
| Outreach services | .3 | 1.4 | 3.9 | 13.5 |
| Agency-based services | .5 | 1.4 | 16.5 | 43.4 |
| Telephone and e-mail services | .4 | 1.1 | .6 | 1.3 |
| Other | ||||
| Outreach services | .0 | .0 | .2 | 1.1 |
| Agency-based services | .0 | .0 | .3 | 1.3 |
| Telephone and e-mail services | .0 | .0 | .0 | .0 |
| Group services | ||||
| Work skills | 349.6 | 657.9 | 40.4 | 73.4 |
| Cognitive rehabilitation | 34.1 | 59.8 | 7.6 | 13.7 |
| Illness management | 18.9 | 39.1 | 14.0 | 29.6 |
| Recreation and physical activities | 219.7 | 446.1 | 5.5 | 12.7 |
| Total (M±SD) | ||||
| Minutes | 789.5 | 1,067.7 | 373.3 | 196.8 |
| Hours | 13.2 | 17.8 | 6.2 | 3.3 |
| Low-fidelity group (N=29) | High-fidelity group (N=22) | |||
|---|---|---|---|---|
| Type of service | M | SD | M | SD |
| Assessment | ||||
| Outreach services | .1 | .5 | .2 | 1.1 |
| Agency-based services | 3.5 | 7.2 | 2.1 | 4.6 |
| Telephone and e-mail services | .0 | .0 | .0 | .1 |
| Individual life counseling | ||||
| Outreach services | .1 | .3 | .2 | 1.1 |
| Agency-based services | 5.1 | 9.8 | 22.2 | 39.0 |
| Telephone and e-mail services | .2 | .7 | 3.3 | 13.1 |
| Individual life skill training | ||||
| Outreach services | 2.1 | 9.5 | 1.1 | 3.4 |
| Agency-based services | 3.9 | 10.2 | 2.3 | 8.2 |
| Telephone and e-mail services | .4 | 2.0 | .3 | 1.1 |
| Counseling for keeping job, on-site job support | ||||
| Outreach services | .0 | .0 | .3 | 1.1 |
| Agency-based services | .4 | 1.2 | 4.7 | 11.4 |
| Telephone and e-mail services | .0 | .0 | .9 | 2.7 |
| Managing services from other agencies | ||||
| Outreach services | .0 | .2 | 2.6 | 12.3 |
| Agency-based services | .2 | .9 | .0 | .0 |
| Telephone and e-mail services | .2 | 1.2 | .2 | .6 |
| Managing services from other professionals in the same agencies | ||||
| Outreach services | .0 | .0 | .0 | .0 |
| Agency-based services | .1 | .3 | 3.8 | 14.9 |
| Telephone and e-mail services | .1 | .3 | .2 | .7 |
| Case meetings | ||||
| Outreach services | .2 | .9 | .5 | 2.1 |
| Agency-based services | .0 | .0 | .3 | 1.6 |
| Telephone and e-mail services | .0 | .0 | .1 | .4 |
| Individual employment services (e.g., job counseling, job development) | ||||
| Outreach services | .2 | 1.0 | 11.2 | 22.8 |
| Agency-based services | .9 | 4.0 | 79.3 | 206.8 |
| Telephone and e-mail services | .0 | .1 | 1.6 | 5.7 |
| Family support | ||||
| Outreach services | .0 | .0 | .0 | .0 |
| Agency-based services | 1.6 | 4.5 | 1.6 | 3.5 |
| Telephone and e-mail services | .0 | .2 | .2 | .7 |
| Other | ||||
| Outreach services | .6 | 1.8 | .2 | 1.1 |
| Agency-based services | .7 | 2.0 | .4 | 1.8 |
| Telephone and e-mail services | .0 | .2 | .0 | .2 |
| Group services | ||||
| Work skills | 67.4 | 162.2 | 24.6 | 49.0 |
| Cognitive rehabilitation | 59.7 | 100.7 | 17.6 | 33.0 |
| Illness management | 49.7 | 100.5 | 4.0 | 10.1 |
| Recreation and physical activities | 127.7 | 325.2 | 95.3 | 158.8 |
| Total (M±SD) | ||||
| Minutes | 325.3 | 439.6 | 281.4 | 395.4 |
| Hours | 5.4 | 7.3 | 4.7 | 6.6 |
Comparison of Service Intensity Between Groups
Table 5 presents the GLM results. Before the participants found jobs, the high-fidelity group was provided significantly more hours of outreach services (B=7.2, 95% confidence interval [CI]=0.2 to 14.1, p=0.043) and agency-based individual services (B=5.7, 95% CI=2.6 to 8.7, p<0.001). Conversely, providers in the low-fidelity group spent more effort on agency-based group services (B=−13.4, 95% CI=−22.3 to −4.5, p=0.003). However, we found no significant between-group differences in service hours for clients who found employment (see Table 4 in the online supplement ), but clients in the low-fidelity group tended to spend more hours in agency-based group services (B=−5.6, 95% CI=−11.8 to 0.6, p=0.076). Sensitivity analyses obtained the same significant trends as the unadjusted models. In addition, supplemental analyses showed that the total hours of service provided to participants before they obtained a job were significantly associated with whether the participants found a job (B=0.1, 95% CI=0.02 to 0.11, p=0.007), but the total hours of service provided during employment were not correlated with work tenure.
| Low-fidelity group (N=29) | High-fidelity group (N=22) | ||||||
|---|---|---|---|---|---|---|---|
| Service and provider | M | SD | M | SD | Ba | 95% CI | p |
| Total | 21.4 | 17.9 | 26.4 | 23.9 | |||
| Employment specialist | 15.3 | 18.9 | 18.4 | 23.9 | |||
| Case manager | 6.1 | 8.9 | 8.0 | 10.9 | |||
| Telephone or e-mail services | .1 | .2 | .5 | .5 | — | — | |
| Employment specialist | .1 | .2 | .3 | .3 | |||
| Case manager | .1 | .1 | .2 | .5 | |||
| Outreach services | 1.9 | 2.6 | 10.5 | 17.9 | 7.2 | .2 to 14.1 | .043 |
| Employment specialist | 1.8 | 2.6 | 10.0 | 18.0 | |||
| Case manager | .1 | .2 | .5 | .9 | |||
| Agency-based individual services | 1.6 | 1.3 | 8.4 | 8.2 | 5.7 | 2.6 to 8.7 | <.001 |
| Employment specialist | 1.3 | 1.3 | 4.9 | 5.7 | |||
| Case manager | .3 | .4 | 3.5 | 7.9 | |||
| Agency-based group services | 17.8 | 15.8 | 7.0 | 7.1 | –13.4 | –22.3 to –4.5 | .003 |
| Employment specialist | 12.1 | 16.5 | 3.1 | 4.0 | |||
| Case manager | 5.7 | 8.7 | 3.9 | 5.1 | |||
a
Results are from the generalized linear model.
Discussion
In this study, we compared the contents and intensity of services provided in low- and high-fidelity supported employment programs for clients diagnosed as having schizophrenia. The findings suggest that high-quality programs, as identified by a high fidelity score, have better vocational outcomes and provide more hours of individual services than low-quality programs.
Supported employment programs with a high fidelity score expended more effort on outreach services and agency-based individual services, particularly job development services, and less effort on group services. In supported employment programs, particularly those using the IPS model, staff members focus on individuals’ preference for job seeking (35). In addition, employment specialists are expected reach out to the community to address job development (18, 36). The high-fidelity group may intentionally increase individual services rather than group services when implementing supported employment services corresponding to individual needs
Both groups provided group services in addition to individual services. The low-fidelity group offered more total service hours per month during the 12-month follow-up than the high-fidelity group, although there was an almost equal number of service uses (days) between the groups. We attribute this seemingly paradoxical finding to the large amount of time spent per day on group services in the low-fidelity group. However, even the high-fidelity group expended about 30% of their efforts on group services. Providing group services even in a high-fidelity supported employment agency appears to be affected by the Japanese mental health care system, which mainly supports group services rather than individual services; thus, this finding may reflect the Japanese context.
Our study is consistent with the findings of McGuire et al. (15), who reported that clients received intensive services in the first 6 months of high-fidelity IPS programs. Their results were similar to those of this study. Because the principles of the IPS model of supported employment include rapid job searches (35), IPS programs appear to provide intensive services during the initial phase of services, regardless of country. In particular, this study’s high-fidelity group received more outreach and individual services before the participants obtained jobs. Considering an association with outcomes, as past studies noted (18, 19, 37), a potential reason that the high-fidelity group had better outcomes is that the participants received more intensive services prior to finding a job.
Contrary to the intensity of services before clients obtained employment, we did not find significant between-groups differences in service intensity during clients’ employment period. Previous studies have demonstrated that the need for follow-up services varies among individuals, particularly those diagnosed as having schizophrenia, as a result of symptom severity or cognitive impairment (15, 21, 22, 38, 39). Because we assessed neither clients’ symptoms nor their cognitive function, this is a limitation of the study. However, the amount of follow-up services may also be influenced by individual preferences and the functional aspect. For example, if a client prefers no follow-up services, service hours are unlikely to increase. Moreover, staff members generally expend effort to solve problems when a client embraces a particular need (e.g., a relationship problem with a colleague) to keep their job rather than when a client’s work is stable. In other words, service quality at the structure level as rated by fidelity may simply not be associated with follow-up service hours. Clarifying the relationship of quality of service structure, amount of service hours, and individual needs and conditions remains an issue for future research.
We recognize that this study had several other limitations. First, collection of service data throughout the study was not perfect. On occasion, staff members did not properly document their service provision on the inventory sheet. However, the mean total service hours per month for the high-fidelity group (10.9±7.7 hours) was similar to that in a Japanese pilot study (11.2±9.4 hours) (17). Second, the sample size was relatively small, although we recruited participants from 13 programs. Hence, the study may have had inadequate statistical power. Third, about 70% of the participating agencies were community social service agencies that did not focus exclusively on people with severe mental illness. Fourth, the quality ratings were performed by using JiSEF. Although this study partly replicated previous findings, its generalizability appears to be limited. Future studies could use the IPS-25 to compare service processes in standard settings in other countries.
Conclusions
This study validated the JiSEF and demonstrates that high-fidelity programs produce better vocational outcomes and provide more individual services before client employment than do low-fidelity programs. Future research should examine fidelity scores (service quality at the structure level), amount of service hours, and individual needs and conditions.
Supplementary Material
File (appi.ps.201900255.ds001.pdf)
- View/Download
- 461.25 KB
References
1.
Suijkerbuijk YB, Schaafsma FG, van Mechelen JC, et al: Interventions for obtaining and maintaining employment in adults with severe mental illness, a network meta-analysis. Cochrane Database Syst Rev 2017; 9:CD011867
2.
Metcalfe JD, Drake RE, Bond GR: Economic, labor, and regulatory moderators of the effect of individual placement and support among people with severe mental illness: a systematic review and meta-analysis. Schizophr Bull 2018; 44:22–31
3.
Richter D, Hoffmann H: Effectiveness of supported employment in non-trial routine implementation: systematic review and meta-analysis. Soc Psychiatry Psychiatr Epidemiol 2019; 54:525–531
4.
Oshima I, Sono T, Bond GR, et al: A randomized controlled trial of individual placement and support in Japan. Psychiatr Rehabil J 2014; 37:137–143
5.
Yamaguchi S, Sato S, Horio N, et al: Cost-effectiveness of cognitive remediation and supported employment for people with mental illness: a randomized controlled trial. Psychol Med 2017; 47:53–65
6.
Hayashi T, Yamaguchi S, Sato S: Implementing the IPS model of supported employment in Japan: barriers and strategies. Psychiatr Rehabil J (Epub ahead of print, Dec 5, 2019).
7.
Craig P, Dieppe P, Macintyre S, et al: Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ 2008; 337:a1655
8.
Moore GF, Audrey S, Barker M, et al: Process evaluation of complex interventions: Medical Research Council guidance. BMJ 2015; 350:h1258
9.
Lockett H, Waghorn G, Kydd R, et al: Predictive validity of evidence-based practices in supported employment: a systematic review and meta-analysis. Ment Health Rev (Brighton) 2016; 21:261–281
10.
Bond GR, Peterson AE, Becker DR, et al: Validation of the Revised Individual Placement and Support Fidelity Scale (IPS-25). Psychiatr Serv 2012; 63:758–763
11.
Sasaki N, Yamaguchi S, Shimodaira M, et al: Development and validation of a Japanese fidelity scale for supported employment. Adm Policy Ment Health Ment Health Serv Res 2018; 45:318–327
12.
Henry AD, Hashemi L, Zhang J: Evaluation of a statewide implementation of supported employment in Massachusetts. Psychiatr Rehabil J 2014; 37:284–288
13.
Margolies PJ, Humensky JL, Chiang IC, et al: Is there a role for fidelity self-assessment in the individual placement and support model of supported employment? Psychiatr Serv 2017; 68:975–978
14.
Margolies PJ, Humensky JL, Chiang IC, et al: Relationship between self-assessed fidelity and self-reported employment in the individual placement and support model of supported employment. Psychiatr Serv 2018; 69:609–612
15.
McGuire AB, Bond GR, Clendenning DR, et al: Service intensity as a predictor of competitive employment in an individual placement and support model. Psychiatr Serv 2011; 62:1066–1072
16.
Cook JA, Lehman AF, Drake R, et al: Integration of psychiatric and vocational services: a multisite randomized, controlled trial of supported employment. Am J Psychiatry 2005; 162:1948–1956
17.
Yamaguchi S, Sato S, Taneda A, et al: Association between amount of supported employment services and vocational outcomes: a process evaluation in participants for cognitive remediation and supported employment services. Jpn J Psychiatr Rehabil 2017; 21:178–187 [in Japanese]
18.
Leff HS, Cook JA, Gold PB, et al: Effects of job development and job support on competitive employment of persons with severe mental illness. Psychiatr Serv 2005; 56:1237–1244
19.
Jones CJ, Perkins DV, Born DL: Predicting work outcomes and service use in supported employment services for persons with psychiatric disabilities. Psychiatr Rehabil J 2001; 25:53–59
20.
Bond GR, Kukla M: Impact of follow-along support on job tenure in the individual placement and support model. J Nerv Ment Dis 2011; 199:150–155
21.
Zito W, Greig TC, Wexler BE, et al: Predictors of on-site vocational support for people with schizophrenia in supported employment. Schizophr Res 2007; 94:81–88
22.
McGurk SR, Mueser KT, Harvey PD, et al: Cognitive and symptom predictors of work outcomes for clients with schizophrenia in supported employment. Psychiatr Serv 2003; 54:1129–1135
23.
Bush PW, Drake RE, Xie H, et al: The long-term impact of employment on mental health service use and costs for persons with severe mental illness. Psychiatr Serv 2009; 60:1024–1031
24.
Harvey PD, Heaton RK, Carpenter WT Jr, et al: Functional impairment in people with schizophrenia: focus on employability and eligibility for disability compensation. Schizophr Res 2012; 140:1–8
25.
Rosenheck R, Leslie D, Keefe R, et al: Barriers to employment for people with schizophrenia. Am J Psychiatry 2006; 163:411–417
26.
Yamaguchi S, Mizuno M, Sato S, et al: Validation of a cut-off score for Japanese version of individualised Supported Employment Fidelity Scale. Jpn J Clin Psychiatry 2018; 47:1431–1438 [in Japanese]
27.
Yamaguchi S, Sato S, Matsunaga A, et al: A process study for implementation of outreach-based case management in a psychiatric day-care: a service quantity analysis. Jpn J Clin Psychiatry 2016; 46:91–102 [in Japanese]
28.
Diagnostic and Statistical Manual of Mental Disorders, 4th ed. Washington, DC, American Psychiatric Association, 1994
29.
Sasaki N, Sato S, Yamaguchi S, et al: Development of a scale to assess motivation for competitive employment among persons with severe mental illness. PLoS One 2018; 13:e0204809
30.
Drake RE, Becker DR, Bond GR: Introducing individual placement and support (IPS) supported employment in Japan. Psychiatry Clin Neurosci 2019; 73:47–49
31.
Gowdy EA, Carlson LS, Rapp CA: Organizational factors differentiating high performing from low performing supported employment programs. Psychiatr Rehabil J 2004; 28:150–156
32.
Tsang HW, Leung AY, Chung RC, et al: Review on vocational predictors: a systematic review of predictors of vocational outcomes among individuals with schizophrenia: an update since 1998. Aust N Z J Psychiatry 2010; 44:495–504
33.
Howard LM, Heslin M, Leese M, et al: Supported employment: randomised controlled trial. Br J Psychiatry 2010; 196:404–411
34.
Metcalfe JD, Drake RE, Bond GR: Predicting employment in the Mental Health Treatment Study: do client factors matter? Adm Policy Ment Health Ment Health Serv Res 2017; 44:345–353
35.
Becker D, Drake RE: A Working Life for People With Severe Mental Illness. New York, Oxford University Press, 2003
36.
Swanson SJ, Becker DR, Bond GR: Job development guidelines in supported employment. Psychiatr Rehabil J 2013; 36:122–123
37.
Metcalfe J, Drake RE, Bond G: The use of standardized discharge in IPS supported employment programs. Psychiatr Serv 2018; 69:476–478
38.
Teixeira C, Mueser KT, Rogers ES, et al: Job endings and work trajectories of persons receiving supported employment and cognitive remediation. Psychiatr Serv 2018; 69:812–818
39.
Mahmood Z, Keller AV, Burton CZ, et al: Modifiable predictors of supported employment outcomes among people with severe mental illness. Psychiatr Serv 2019; 70:782–792
Information & Authors
Information
Published In
History
Received: 19 May 2019
Revision received: 17 September 2019
Accepted: 24 October 2019
Published online: 3 January 2020
Published in print: May 01, 2020
Keywords
Authors
Competing Interests
The authors report no financial relationships with commercial interests.
Funding Information
JSPS KAKENHI: Grant-in-Aid for Young Scientists [B], 16K21661: D
Ministry of Health, Labour and Welfarehttp://dx.doi.org/10.13039/501100003478: H26-002: A study on facilitating employment for pe
This study was funded by the Ministry of Health, Labor, and Welfare in Japan (H26-002, “A Study on Facilitating Employment for People With Mental Illness”) and JSPS KAKENHI (Grant-in-Aid for Young Scientists [B], 16K21661, “Development of Japanese Version of IPS/Supported Employment Fidelity Scale and Evaluation System”).Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBGet Access
Login options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).