Permanent Supportive Housing for Transition-Age Youths: Service Costs and Fidelity to the Housing First Model

The PSH programs in California provide individuals with serious mental illness who are homeless or at risk of homelessness with subsidized permanent housing and multidisciplinary team–based services with a focus on rehabilitation and recovery. PSH services follow either an intensive case management model or a multidisciplinary treatment team model (22). Clients are recruited through outreach and referrals from psychiatric hospitals, emergency rooms, other mental health programs, county agencies, jails, shelters, rescue missions, and the street. Most PSH programs deliver services to clients in real-world settings: in their homes, workplaces, and other places in the community chosen by the client or deemed of therapeutic value by staff. Crisis intervention services are available 24 hours a day, seven days a week.

I used administrative data (described below) to identify 2,609 youths ages 18 to 24 with a diagnosis of serious mental illness (defined as schizophrenia, bipolar disorder, or major depressive disorder) who enrolled in PSH programs between January 1, 2005, and June 30, 2009. Propensity score methods were used to identify a comparison group of youths with serious mental illness with demographic and clinical characteristics and health service use profiles similar to those of PSH clients (23,24). I identified 42,112 youths with serious mental illness who were receiving outpatient mental health services during the same time frame as youths enrolling in PSH programs. Because these youths did not have participation dates corresponding to youths in PSH programs, I randomly selected a participation date from the days on which an outpatient mental health service was received. Using logistic regression, I estimated a propensity score of PSH participation based on prior-year service utilization and participation month, as well as on age, gender, race-ethnicity, clinical diagnosis, comorbid substance use disorder, Medicaid coverage, and county of residence. I identified a matched comparison group by using nearest-neighbor matching (25). This approach has been used to identify comparison groups for studies of PSH programs for adults (5,26,27).

Data on mental health service utilization and costs were derived from multiple administrative data sets from the State of California. The Department of Mental Health provided data on PSH enrollment, demographic characteristics, clinical diagnoses, and mental health service utilization. The Office of Statewide Health Planning and Development provided data on inpatient admissions (including both psychiatric admissions and those for general medical conditions) and emergency department admissions (also related to both mental and general medical conditions). The Department of Health Care Services provided information on service costs. Merging these data involved matching specific services across multiple files in order to remove the overlap in reporting and avoid double-counting of services. The resulting data set captured all psychiatric services and their costs (except for state hospitals and jails) as well as nonpsychiatric inpatient and emergency department admissions.

Service use and costs were calculated for one year before and one year after enrollment in the PSH program. I calculated costs for the following categories of service: inpatient, crisis and residential services (including services provided by crisis residential facilities, psychiatric health facilities, residential facilities, emergency departments, and institutions of mental disease), and mental health outpatient (including assessment, medication management, rehabilitation, and therapy). Utilization and cost data were available from January 1, 2004, through June 30, 2010. Thus clients had a full year of exposure to services in their pre- and postentry periods.

Fidelity was measured at a point in time, from June through November 2010, with the self-report Housing First Fidelity Survey (28). Among the 58 counties in California, 53 (91%) implemented PSH programs under the MHSA, and of these counties, 23 (43%) participated in survey. Among participating counties, 93 of 135 FSP programs (69%) responded to the survey. The survey measures fidelity to the Housing First model across two factors and five domains. One factor measures fidelity with respect to housing choice and structure, separation of housing and services, and service philosophy. A second factor measures fidelity with respect to service array and team structure. Factor scores were used to rank programs by fidelity: the top 20% of programs were designated as high fidelity and the bottom 20% were designated as low fidelity; the remaining were mid-fidelity (29).

Health service costs were analyzed by using a quasi-experimental, intent-to-treat, difference-in-differences (DID) design with a propensity score–matched contemporaneous control group (30). A DID estimator calculates the treatment effect by estimating the pre-post difference and accounting for possible confounding time trends by subtracting the observed pre-post difference values from a control group. The intent-to-treat design included all PSH participants even if they were discharged from the program during the follow-up period. Propensity score matching helped to ensure the validity of the key assumption of the DID design: comparable time trends between the PSH participants and an otherwise comparable control group. Health service use by level of fidelity to the Housing First model was analyzed using a pre-post design among youths enrolled in PSH programs that responded to the Housing First Fidelity Survey.

I used generalized linear models to estimate health service costs and service outcomes. I used two-part models to analyze inpatient and crisis and residential service costs. The two-part model is commonly used to estimate health care costs when the dependent variable is nonnegative and when its distribution is noticeably skewed and kurtotic (with a heavy right-hand tail) (31). I used logistic regression to estimate the probability of any use and a model based on a gamma family with a log link function to estimate costs conditional on use of services. I used a single model based on a Poisson family with a log link function to estimate mental health outpatient costs. Total costs were estimated as the sum of the estimated component costs. I used zero-inflated negative binomial regression models to estimate the number of outpatient visits (32–34). I selected these specific distributions on the basis of standard tests for assessing alternative generalized linear and transformed models (35–37). I assessed goodness of fit by using a modified Hosmer-Lemeshow test and a Pregibon’s link test (38,39).

The primary variables of interest in the models comparing PSH participants with the propensity score–matched control group were indicator variables for participation in the PSH program for the postperiod and for the interaction between the PSH and the postperiod. The primary independent variables of interest in the models comparing PSH participants by level of fidelity to the Housing First model were indicator variables for levels of fidelity and for interactions between levels of fidelity and the postperiod. In all models, I included age, gender, race-ethnicity, clinical diagnosis, comorbid substance use disorders, and Medi-Cal coverage as additional control covariates.

Incremental effects associated with PSH and level of fidelity were standardized to the underlying population characteristics. For the models comparing PSH participants with the propensity score–matched control group, I calculated pre- and postperiod estimates for PSH clients, pre- and postperiod estimates for clients in the control group, and the difference between these estimated pre-post differences (DID estimate). For the models comparing PSH participants by level of fidelity, I computed standardized preperiod, postperiod, and difference estimates by level of fidelity. These estimates were based on the assumption that a program had the same level of fidelity for each of the two factors of fidelity to Housing First. Standard errors were calculated by using the nonparametric bootstrap with clustering at the program level, and p values were computed by using the percentile method from the empirical distributions of the results from 1,000 replicates (40). All analyses were conducted in Stata, version 13 (41).

The University of California, San Diego, Human Research Protections Program, the State of California Committee for the Protection of Human Subjects, and the Office of Statewide Health Planning and Development approved the use of these data for the purpose of this study in accordance with the privacy rule of HIPAA.

Factor scores were used to rank PSH programs by fidelity for each of the two factors. Of the 93 PSH programs surveyed, 77 (83%) were located in counties that provided service utilization data and of these, 63 (82%) enrolled transition-age youths. Table 2 shows PSH program characteristics of these 63 programs at low, medium, and high levels of fidelity to the Housing First model. By design, high-fidelity programs were more likely than mid- or low-fidelity programs to meet fidelity thresholds for most items in the five domains. Exceptions included the two items in the housing choice and structure domain and opportunities for community employment and volunteering and program provision of three core social integration services in the service array domain. Low-fidelity programs were less likely than mid- and high-fidelity programs combined to offer these services (p<.05 for each, data not shown).

Estimates of annual standardized costs are shown in Table 3. Inpatient costs and costs for crisis and residential services declined to a greater extent for the control group than for youths in PSH programs. As a result, the DID estimates show that inpatient costs increased by $1,088 (p=.046), costs for crisis and residential services increased by $1,271 (p=.036), and mental health outpatient costs increased by $10,979 (p<.001) for youths in PSH programs compared with the control group; as a result, total service costs were $13,337 (p<.001) higher for youths in PSH programs compared with the control group.

Table 4 shows the standardized probability of inpatient admission and the standardized number of outpatient mental health visits by level of program fidelity. There were no significant differences in the probability of inpatient admission by level of program fidelity in the preperiod. The probability of admission decreased by 9.2 percentage points (p=.004) among youths in mid-fidelity programs and by 14.9 percentage points (p<.001) among youths in high-fidelity programs. Youths in low-fidelity programs had more outpatient mental health visits in the preperiod than youths in high- and mid-fidelity programs (p<.001 for between group differences, not shown). Patients in high-fidelity programs had the largest increase in the number of visits, followed by clients in mid- and low-fidelity programs: 63.8 versus 43.7 and 26.9, respectively (p<.001 for between group differences, not shown).

Table 5 shows standardized total costs by level of program fidelity. Patients in high-fidelity programs had the largest increase in costs, followed by clients in mid- and low-fidelity programs: $17,610 versus $7,224 and $4,575, respectively (p<.001 for between-group differences, not shown).