Fidelity to the Housing First Model and Effectiveness of Permanent Supported Housing Programs in California

The FSP programs in California provide individuals with serious mental illness who are homeless or at risk of homelessness with subsidized permanent housing and multidisciplinary team-based services with a focus on rehabilitation and recovery. FSP services typically follow either an intensive case management model or a modified assertive community treatment model (14). Clients are recruited through outreach and referrals involving psychiatric hospitals, emergency rooms, other mental health programs, county agencies, jails, shelters, rescue missions, and service providers working on the streets. Most FSPs deliver services to clients in real-world settings, such as their home or workplace and other places in the community chosen by the client or deemed of therapeutic value by staff. Crisis intervention services are available 24 hours a day, seven days a week.

Because there was no existing conceptual framework to describe FSP practices, a framework was developed to compare FSP practices with the Housing First model, a benchmark program sharing similar goals, vision, and structure. Housing First programs provide immediate access to affordable, permanent, scatter-site housing with tenancy rights and access to team-based services designed in accordance with a recovery-oriented service philosophy, which draws heavily on the psychosocial rehabilitation model (15). Key elements of this approach are consumer choice, self-determination, and independence; the active use of harm reduction, motivational interviewing, assertive engagement, and person-centered planning by program staff; and the absence of coercive practices.

Fidelity was measured by using the Housing First Fidelity Survey (16). The survey measures fidelity to the Housing First model across two factors and five domains. One factor measures fidelity with respect to housing choice and structure, separation of housing and services, and service philosophy. A second factor measures fidelity with respect to service array and team structure. This article focuses on the first factor because its three associated domains are closely associated with the FSP programs’ approach to housing.

The California Department of Mental Health Data Collection and Reporting (DCR) system was used to identify FSP clients and to summarize days spent in various residential settings from one year preenrollment to one year postenrollment. The sample included individuals with serious mental illness (defined as schizophrenia or schizoaffective disorder, bipolar disorder, or major depressive disorder) who were enrolled in participating FSPs between January 1, 2005, and June 30, 2009. Clients who were not enrolled for at least 180 days were excluded from the analyses. Information about the clients’ history of housing was identified from an FSP assessment form, and changes in residential status over time were derived from key-event tracking forms (12). Residential settings included homeless (includes living in one’s car), emergency shelter, and temporary housing (includes living with a friend but not paying rent); justice system; congregate or residential (group arrangements, most commonly group living homes and board and care residences); apartment or single-room occupancy (SRO) hotel; residences of parents, family, and others; and unknown. Thus these living arrangements represent a range of options, from homelessness to shared, highly supervised, and structured congregate settings and more independent settings, such as apartments and SRO hotels.

Changes in days in residential settings in the overall sample and by level of program fidelity were examined by using paired t tests. Factor scores derived from the Housing First Fidelity Survey (16) of 93 FSP programs were used to rank the programs on fidelity to the Housing First model with respect to housing choice and structure, separation of housing and services, and service philosophy. On the basis of our knowledge of the programs and an examination of natural cut points, programs with factor scores in the top 20% were designated as having high fidelity to the Housing First model, and programs with factor scores in the bottom 20% were designated as having low fidelity; the remaining programs were designated as having mid-fidelity. If a client was enrolled in an FSP for fewer than 365 days in the postenrollment period, the number of days in each residential setting was first annualized by dividing by the days enrolled in the FSP and multiplying by 365.

Changes in days in residential settings by level of program fidelity were estimated by using a series of generalized estimating equations (GEEs) specified with a Gaussian distribution and identity link, an exchangeable correlation matrix to account for correlated errors within FSP programs, and an exposure offset to adjust for differences in enrollment in the postenrollment period (17). GEEs were estimated for each residential setting, where the dependent variable was the change in the number of days in the residential setting from the preenrollment to the postenrollment period and the main independent variables of interest were indicator variables for each level of fidelity (the intercept was suppressed). Two sets of GEEs were estimated. One set controlled for the following individual characteristics: age, gender, race-ethnicity, clinical diagnosis, diagnosis of a substance use disorder, and Medicaid coverage. A second set controlled for individual characteristics and days in the residential setting in the preenrollment period. All analyses were conducted in Stata, version 12 (18).

Site visits were conducted at 20 programs that were purposefully sampled by using a maximum variation strategy (19) to provide geographic, political, and economic diversity as well as a wide range of scores on the Housing First Fidelity Survey (13,16). Sampling was a two-stage process. Participating counties were selected from southern, central, and northern California, including coastal and inland and urban and rural counties. Within counties, programs were selected to maximize the range of fidelity scores. Site visits involved observation, interviews with program staff, and focus groups with clients.

Summaries of audio recordings of client focus groups created during the site visits at the five highest and five lowest fidelity sites were reviewed for insights into clients’ experiences. The qualitative data were used to complement the quantitative data by providing a depth of understanding not available by using the quantitative data alone (20). A template or matrix approach (21,22) to text analysis was used to identify specific housing topics that were discussed by each focus group and to compare how the programs addressed topics discussed by both sets of focus groups.

A total of 86 FSPs with 7,186 clients provided DCR data for this study; 602 clients were excluded because they had been enrolled in the FSP for fewer than six months. Demographic and clinical characteristics of 6,584 FSP clients by level of program fidelity are shown in Table 1. The mean±SD age was 40±14; 2,988 (45%) were female; 2,537 (39%) were non-Latino white, 967 (15%) Latino, 901 (14%) African American, 241 (4%) Asian, and 1,938 (29%) other or unknown race-ethnicity; 3,973 (60%) had a diagnosis of schizophrenia, 1,406 (21%) bipolar disorder, and 1,205 (18%) major depressive disorder; 3,355 (51%) received a diagnosis of a substance use disorder; and 4,855 (73%) had Medicaid coverage prior to enrollment in the FSP. Compared with clients in lower fidelity programs, clients in high-fidelity programs were more likely to be age 60 or older, more likely to be non-Latino white, less likely to have schizophrenia, and more likely to have a substance use disorder.

Table 2 shows FSP program characteristics by level of program fidelity. In general, high-fidelity programs were more likely than low-fidelity or midfidelity programs to meet fidelity thresholds for housing choice and structure and separation of housing and services. High-fidelity and midfidelity programs were more likely than low-fidelity programs to meet fidelity thresholds with respect to service philosophy.

Qualitative data from the focus groups revealed differences between high- and low-fidelity FSPs in client choice in housing and the degree to which client-driven goals were considered in determining housing placement. Clients of high-fidelity FSPs described being given choices among apartments and locations: “[The FSP] gave me opportunities to look at different places, but I picked the one on [Name] Street because I feel that’s what I wanted.” In contrast, clients in low-fidelity programs reported that they were simply assigned to housing by the FSP: “There was really no decision. It’s what I was offered: ‘You can stay here or you can stay on the streets.’ . . . It was just offered to me, board and care. . . . I don't know if I had any other choices.”

Clients of high-fidelity programs reported that FSPs helped them to find housing that met their individual needs or helped them work toward their personal goals. Thus a client described the FSP’s helping her find housing that would support her goal of reuniting with her family: “I’ve been in [City] for over a year and a half [in a one-bedroom apartment]. . . . I specifically requested [City] because part of my recovery was to re-establish my relationship with my kids. And my kids are in [City], so that’s why I’m still in [City].” In another high-fidelity program, a client talked about how the FSP found him an apartment that was convenient to transportation and the downtown business district of the city. As he put it, “It's a small apartment, very small . . . it’s 300 square feet, but I've got a refrigerator; it's nice. . . . I’m looking for work and it’s central . . . that was one of the considerations so that I would be able to find work and get to it.” Participants in the focus groups at the low-fidelity programs did not tell any stories that reflected the role of self-determination and client-driven goals in housing placement.

Table 3 shows the average number of days that FSP clients spent in various living situations in the year before and after enrollment, both overall and by level of program fidelity. Overall, days spent homeless declined by 47, days spent in justice system settings declined by 10, and days spent with parents or family members declined by 11. Days spent living independently increased by 26, and days spent in congregate or residential settings increased by 47 (p<.001 for each).

There were important differences in days spent in residential settings by level of fidelity. Notably, the mean number of days spent homeless in the preenrollment period was lowest at low-fidelity programs, followed by midfidelity and high-fidelity programs (46, 65, and 101 days, respectively). Declines in days spent homeless were also lowest at the low-fidelity programs compared with midfidelity and high-fidelity programs (declines of 29, 40, and 77 days, respectively). Conversely, the number of days spent in justice system settings in the preenrollment period was highest at low-fidelity versus midfidelity and high-fidelity programs (38, 22, and 11 days, respectively), and declines in justice days were greatest among low-fidelity versus midfidelity and high-fidelity programs (declines of 21, ten, and five days, respectively). The number of days spent living in apartments was nearly identical across program types during the preenrollment period, but during the postenrollment period, it increased by 67 days at high-fidelity programs compared with an increase of 12 days at midfidelity programs and a decrease of nine days at low-fidelity programs.

Table 4 shows annual changes in days spent in various residential settings by level of program fidelity, adjusted for individual characteristics of FSP clients. The number of days spent homeless declined at all programs, but the declines were greatest at high-fidelity FSPs compared with midfidelity and low-fidelity FSPs (declines of 87, 49, and 34 days, respectively; p<.001). The number of days spent living independently in an apartment or SRO hotel increased by 44 at high-fidelity FSPs but declined by 21 at low-fidelity programs (p<.001).

Table 5 shows annual changes in days spent in various residential settings by level of program fidelity, adjusted for individual characteristics and days in the residential setting during the preenrollment period. The difference between the programs in the number of days homeless narrowed considerably (declines of 63 days, high fidelity; 52 days, midfidelity; and 53 days, low fidelity; p=.039). The number of days spent living independently in an apartment or SRO hotel increased by 33 at the high-fidelity FSPs but declined by 30 at the low-fidelity programs (p<.001).

We conducted a post hoc analysis to investigate the finding that clients at low-fidelity FSPs spent more days in jail during the preenrollment period compared with clients at midfidelity and high-fidelity FSPs. This finding resulted from a single FSP (N=201) that was oriented toward adults exiting the justice system. This FSP had low fidelity (offering mostly emergency, short-term, or transitional housing with readiness requirements and requiring participation in treatment), and its clients spent a high number of days (93±115 days) in the justice system in the year prior to enrollment and had large reductions in days (decline of 71±119 days) in the justice system during the year postenrollment.