Study group and design
All study participants met the enrollment criteria of severe mental illness, unemployment, willingness to give informed consent, and absence of memory impairment or medical illness that would have prevented them from being employed or participating in research interviews. Individuals were considered to have a severe mental disorder if they met criteria for schizophrenia, schizoaffective disorder, bipolar disorder, recurrent major depression, or borderline personality disorder and had two years of impaired role functioning. Participants were recruited at a mental health agency in southeast Washington, D.C., that serves persons who need intensive case management, usually because their psychiatric disorders are complicated by homelessness, a comorbid substance use disorder, or HIV infection.
During an 18-month period from January 1994 to July 1995, all agency clients were encouraged to attend informational groups, and interested attendees were screened for eligibility by the project director (
9). Random assignment was stratified according to work history—more than one year of continuous employment in a previous job. A total of 76 participants were assigned to IPS and 76 to EVR. However, two of the IPS participants dropped out during the study. Nonparametric tests revealed a near-significant trend of less hospitalization for the EVR group in the period before the study. Additionally, we reviewed our data and excluded an "outlier" from the IPS group because IPS was not considered to be the cause of this person's costs or effects. He had not engaged in competitive work in the year before the study, was hospitalized for more than 300 days during the study period, and had virtually no participation in the IPS program.
Interventions
Individual placement and support was developed to provide supported employment services for people with severe mental disorders in community mental health centers (
10). IPS integrates mental health and vocational services with the addition of an employment specialist to the multidisciplinary case management team. IPS employment specialists help clients search for jobs rapidly. After employment is secured, the specialists provide individualized, follow-along supports such as counseling, transportation, or intervening with an employer. Services are provided as needed and with no limits. Three employment specialists were hired to implement IPS. The IPS program was implemented according to a manual (
11) and was monitored by the research team. Ratings indicated that the IPS program consistently scored within the "high fidelity" range on the IPS Fidelity Scale (
12).
The comparison condition, EVR, consisted of vocational rehabilitation services provided by several well-established agencies recommended by the District of Columbia Rehabilitation Services Administration. The program was considered "enhanced" because an extra vocational rehabilitation counselor was placed in the Rehabilitation Services Administration office to ensure rapid and assertive linkage with service vendors to avert the problem of dropout during the referral process. The vocational rehabilitation counselor monitored participants monthly, and if clients were dissatisfied with the program to which they were assigned or dropped out of vocational services, the counselor attempted to link the participants with another agency.
It is important to note that all of the EVR agencies endorsed competitive employment as their goal but used stepwise approaches that involved prevocational experiences, primarily paid work adjustment training in sheltered settings.
Costs
We adopted the cost perspective of a hypothetical single payer of all treatment costs (
17). Like most care systems, the one we studied involves multiple public payers and agencies, but focusing on any one of those entities would limit the generalizability and relevance of the study for other systems of care. All costs were calculated in fiscal year 1995 dollars. Overall costs were measured as the sum of direct costs for mental health and vocational rehabilitation services for the entire 18-month study period.
Mental health costs had both inpatient and outpatient components. Outpatient mental health costs included costs for medication management, intensive case management, group therapy, and family therapy. Overall costs were calculated by multiplying the number of hours of service by the hourly unit cost estimates for each service type.
Because the same agency was responsible for outpatient services to both IPS and EVR participants, unit costs for mental health services were the same for both study conditions. The unit costs for outpatient services were calculated by using data abstracted from the agency's audited 1995 financial statements. Direct and overhead costs for medication management, intensive case management, and family therapy were divided by the estimated number of hours of service reported in daily service logs completed by all staff (
Table 1).
Estimates of the costs for group therapy were derived from the unit cost for intensive case management—$102.87. Staff case managers conducted groups of eight patients for 1.5 hours. Thus the unit cost for group therapy was $19.29, or $102.87 multiplied by 1.5 hours and divided by eight.
The unit cost for inpatient mental health services was calculated by multiplying the number of hospital days for each participant by an average hospital cost per day derived from Medicare Cost Report data for two of the most heavily used hospitals. The average hospital cost per day—$477.89—was determined by dividing the total cost for psychiatric beds by the number of psychiatric bed days in fiscal year 1995.
Vocational rehabilitation costs were calculated differently for each program. For IPS participants, IPS workers completed service logs indicating the number of service hours each participant received. The number of hours per participant was multiplied by an hourly unit cost. The unit cost was computed by using the two-year ratio of overall costs for IPS vocational rehabilitation services reported in cost statements for fiscal years 1994 and 1995, or $209,000+$209,000= $418,000, divided by the number of hours spent with clients reported in service logs for both years, 3,240.3+2,677.1=5,917.4. A comparison of the resulting unit cost—$418,000 divided by 5,917.4= $70.64—with year-specific unit costs (that is, fiscal year 1994 versus fiscal year 1995) indicated only modest start-up costs. Although the study continued for part of 1996, the same level of IPS activity did not remain the same. Fiscal year 1996 data reflect a reduction in the level of services delivered to participants.
For EVR participants, vocational rehabilitation costs were the sum of three costs: the cost of the coordinator, the cost to the Rehabilitation Services Administration, and additional assessment costs routinely used by the Rehabilitation Services Administration, such as the costs of medical and psychological evaluations. The coordinator cost included both the coordinator and supervision. The coordinator had a senior supervisor from the Rehabilitation Services Administration and worked to match EVR participants with community vocational rehabilitation programs (vendors). We considered the annual salary, fringe benefits, and overhead for the coordinator and for one-seventh of the supervisor's time, because the supervisors typically worked with seven frontline workers. The estimated annual cost for the coordinator was $61,353. We assumed that over a three-year period, the cost for the coordinator was three times that amount, or $184,059.
The coordinator logged all time dedicated to each study participant. Using methods similar to those used to calculate IPS costs, we developed an hourly unit cost based on the coordinator's productivity in the first 18 months of the study to avoid an underestimate of the coordinator's productivity in the third year. During the first 18 months, EVR participants used 1,247 hours of vocational rehabilitation services that were provided by the coordinator. Thus we estimated that over the three years, the coordinator provided 2,494 hours of services (2×1,247). We then divided the three-year cost by the estimated three-year service use to obtain a unit cost for the coordinator of $73.80 ($184,059 divided by 2,494). Individual use of each type of service was multiplied by the unit cost to estimate the total costs for enhanced vocational rehabilitation. The additional start-up costs were modest, as they were for IPS.
For the second component of the costs for EVR, we used actual payments from the Rehabilitation Services Administration to individual vendors for services provided to the EVR group. The third EVR cost component included additional assessments such as medical and psychological evaluations arranged by the Rehabilitation Services Administration. These were assigned a nominal value of $50 per service, which reflected the administration's estimate of its costs for these services.
Data analysis
We compared the costs and effects of IPS and EVR. Because of the skewness of the data and the small sample, nonparametric statistical tests—the Wilcoxon rank sum test—were used. Overall costs were further investigated with two multiple linear regression analyses. The more parsimonious regression model estimated the effect of vocational intervention on the natural logarithm of overall costs while adjusting for the prestudy hospitalization rate. We also estimated overall costs by using a richer model that adjusted for vocational intervention, age, gender, race, substance abuse, and a diagnosis of schizophrenia as well as for the prestudy hospitalization rate.
To estimate the cost-effectiveness of IPS compared with EVR, we constructed an incremental cost-effectiveness ratio (ICER). The ICER is the statistic of interest in cost-effectiveness analysis and has been used in economic evaluations of other mental health programs (
17,
18,
19,
20). The ICER was computed by dividing the difference in average cost for each program by the difference in average effectiveness. The ICER we used estimated the increased cost associated with gaining a one-unit improvement in effect over 18 months. Because we considered three effects—hours of competitive work, weeks of competitive work, and average combined earnings—we generated three ICER estimates. We used bootstrapping methods to create 5,000 ICER estimates that were plotted on cost-effectiveness planes (scatterplot graphs) (
19,
20,
21). In this way, we were able to assess the degree of uncertainty associated with our estimates of the ICERs. Details of the bootstrapping process are available from the authors.