Housing Programs for Homeless Individuals With Mental Illness: Effects on Housing and Mental Health Outcomes
Abstract
Objective:
This systematic review analyzed the best available research in the United States on permanent supportive housing programs for homeless individuals with mental illness and the effect of these programs on housing status and mental health. It updates older and broader reviews that included weaker studies or those that did not analyze permanent housing as an input and housing and mental health as primary outcomes.
Methods:
The literature search (1980–2013) yielded 14 studies (randomized controlled trials and quasi-experimental studies).
Results:
The studies found that a majority of participants placed in experimental housing programs with case management support remained in housing for at least one year or experienced more days housed than homeless relative to a comparison group. Although this finding is in line with previous literature reviews on permanent supportive housing, this analysis found limitations in each of the 14 reviewed studies, such as attrition, selection and response bias, imprecise definitions and implementation of housing programs, and a lack of appropriate controls. Only three of the reviewed studies reported using a housing fidelity assessment tool to test whether the housing intervention was faithful to theoretical standards, and conceptions and implementation of housing varied widely across studies, threatening internal and external validity.
Conclusions:
Pitfalls in the best available studies on permanent supportive housing programs in the United States limit the ability of research to inform the policy goal of ending chronic homelessness and demonstrate a need for further experimental research upon which to make funding and policy decisions, especially in light of prioritized federal funds.
Homelessness among individuals with severe mental illness has been called “the most pervasive manifestation of the failure of public policy” (1). The U.S. Department of Housing and Urban Development (HUD) estimated that 610,042 people were homeless on a single night in January 2013. More than seven million poor Americans—among the more than 45 million Americans living in poverty—are at risk by living “doubled up” with friends or family (2,3). The rate of mental illness in the homeless population is higher than that in the general population (4). A 2008 meta-analysis found that in the homeless populations of western countries rates of psychosis ranged from 3% to 42%; the upper range for alcohol or drug dependence was as high as 50% (5).
In recent years, federal officials and treatment experts have touted permanent supportive housing as the solution to ending chronic homelessness among individuals with severe mental illness (6). Permanent supportive housing is loosely defined as subsidized housing coupled with supportive services. It has been implemented in a variety of ways, from scatter-site apartments visited by case managers to clustered housing with onsite staff.
No standardized model for housing homeless individuals with mental illness exists to guide research or policy (1,7). Supportive housing programs vary widely, in part because of financial and infrastructure constraints, such as a lack of affordable housing, and philosophical differences that have persisted for decades about how best to house and treat this population (1,8). Housing studies involving this population are plagued by vague or conflicting definitions and conceptualizations of supported housing, hampering comparative analysis (9,10). Despite such limitations, permanent supportive housing has overtaken so-called “transitional” or continuum of care (CoC) housing programs as the preferred public policy strategy for housing individuals who are chronically homeless (11). As the primary funding source for homeless services nationwide, HUD requested that communities applying for CoC grants during the 2013–2014 funding cycle prioritize individuals who were chronically homeless for permanent supportive housing that is based on a Housing First approach, because research has shown that this model is more effective and less costly than other approaches (12).
In CoC or transitional housing, progress from short-term housing into independent living in permanent supportive housing is predicated on participation in mandatory psychiatric treatment and sobriety goals established by service providers (13). The only major codified difference between transitional and permanent supportive housing concerns timing: HUD-funded transitional housing is limited to 24 months, whereas permanent supportive housing has no limitation.
Although practical differences may be more difficult to determine, philosophical differences between transitional housing and the Housing First approach run deep. Unlike transitional housing programs, Housing First asserts a right to housing without a requirement that homeless individuals take medication, receive psychiatric treatment, or abstain from using drugs and alcohol as a condition of receiving housing—requirements widely criticized as ineffective by homeless individuals and service providers (13,14). In announcing the agency’s decision to prioritize funding for Housing First programs, HUD in 2013 defined the model as “housing assistance that is offered without preconditions (such as sobriety or a minimum income threshold) or service participation requirements” with “rapid placement and stabilization in permanent housing” as primary goals (12). Permanent supportive housing programs do not necessarily operate with the degree of tenant autonomy ascribed to the Housing First model pioneered by Pathways to Housing in New York, in part because of disagreement over how much independence is appropriate for this population (10). Still, policy makers have partly attributed double-digit declines in chronic homelessness in recent years to the success of permanent supportive housing, especially Housing First–type programs (11,15).
The policy emphasis on permanent supportive housing is not based on a large or definitive base of research and has been heavily influenced by political and social considerations (1). The landmark McKinney-Vento Homeless Assistance Act of 1987 included funding for permanent supportive housing and largely predated experimental research on such programs, which employ a variety of housing and supports (1,10).
Although positive research outcomes helped build national support for the Housing First model, it also gained legitimacy because researchers and policy makers framed chronic homelessness as an economic problem with a market-based solution. By defining the problem as an affliction among individuals with mental illness who are frequent and expensive users of public services, advocates spread the message that Housing First could address a public eyesore while saving communities money (16). Advocates claim that housing is cheaper than emergency care, although experimental research has yielded modest savings among programs that do not cover housing costs or among programs that inflate estimated savings in public service expenses by calculating average costs rather than marginal costs (17–19). A recent quasi-experimental study in California found that mental health costs and outpatient visits increased among participants in permanent supportive housing programs, which may have resulted from increased contact with service providers (20).
Several literature reviews have analyzed research on the effects of housing programs on homelessness (1,8,21–27). These broader reviews include methodologically weaker studies and studies that analyzed inputs other than permanent housing. This article updates these reviews by summarizing findings from the most recent and credible literature and restricting results to randomized controlled trials and quasi-experimental studies in the United States that analyzed the effect of supportive housing programs on housing or mental health outcomes.
Methods
This analysis reviewed results of the best available research in the United States on permanent housing programs for homeless individuals with mental illness and the effect of these programs on treatment outcomes, including housing status and mental health. The review was limited to studies in which permanent housing was offered as an intervention component and research input, including subsidized housing offered through housing vouchers. A search of indexed literature from January 1980 through December 2013 using the terms “homeless,” “experimental,” “randomized,” “housing,” “supported housing,” and “supportive housing” was conducted, limiting findings to randomized controlled trials and quasi-experimental studies. These searches were cross-referenced against multiple databases, including PsycINFO and Thomson Reuters Web of Knowledge. The search yielded 12 primary studies (7,8,13,17,28–35), and two secondary analyses (36,37), involving more than 7,400 homeless participants with mental illness.
The review excluded research that did not include permanent housing, such as studies of transitional housing, inpatient treatment of substance use disorders, respite care, day treatment, case management, drop-in centers, and outreach programs. Also excluded were studies comparing housing programs that were too similar to warrant analysis of differences in housing outcomes and those examining the contribution of housing to outcomes other than housing and mental health, such as cost-effectiveness, hospitalization, and social service use. Studies that examined housing as an output, such as the effect of substance use disorders or mental illness on housing stability, were excluded, as well as studies involving homeless individuals who were not mentally ill or individuals with mental illness who were not homeless. This body of research thwarts a meta-analysis because it lacks sufficient data to compute effect sizes and because it adopts incompatible interventions or weak methodologies that make comparative analysis difficult. Findings are analyzed in narrative form.
Results
The studies found that a majority of participants placed in experimental housing programs with case management support remained in housing for at least one year or experienced more days housed than homeless relative to a comparison group. This finding is in line with literature reviews concluding that most homeless individuals with mental illness stay in housing regardless of housing program type (24), although results are mixed when programs are compared with each other (1,25).
Twelve of the 14 studies reported housing-related outcomes, and 11 of the 12 reported statistically significant results supporting a hypothesis that the preferred housing intervention outperformed a control condition (7,8,13,17,29–35). The remaining study set out to settle the debate between the Housing First and CoC models and did not articulate a hypothesis to be tested (28). It found no difference in housing tenure between experimental and control conditions.
Of the 11 studies with statistically significant results supporting a hypothesis in favor of a preferred housing intervention, two did not guarantee housing for participants in the experimental group, thwarting comparisons to the majority of studies in this review that did guarantee housing (7,29). When interventions were compared with control conditions, the intervention in one of these studies did not reduce homelessness (29). In a third study, the intervention did not reduce homelessness among individuals with low to moderate psychiatric and substance use disorder symptoms (30). A fourth study employed experimental and control conditions that were so difficult to distinguish as to yield results of limited practical value for policy makers (8). The remaining seven studies showed significant differences in housing stability favoring experimental over control conditions, but, like the others, they suffered from methodological challenges (13,17,31–35).
The studies defined housing outcomes in various ways, limiting external validity. Seven reported as a primary outcome the percentage of participants housed at the end of the study period, although some also measured other housing indicators, such as length of time to achieve stable housing (8,13,28,29,33–35). Three studies reported the proportion of time spent in stable housing versus homeless (30–32). Two studies reported the number of days in which participants were housed versus homeless (7,17). Studies used a variety of methods to obtain data on housing status.
Seven of the 14 studies reported mixed clinical and substance use outcomes (7,8,31–33,36,37). One of these studies found support for a hypothesis that the experimental housing condition outperformed a comparison or control condition by reducing psychiatric symptoms (8). Another study reported that the experimental housing condition was associated with a reduction in substance use (36). Another found no difference in substance use between persons in experimental and control conditions because substance use declined in both conditions (37). Four studies found that the preferred housing condition did not yield any advantage in clinical outcomes over the comparison or control condition, either because no improvements were found (7,32,33) or because both experimental and comparison groups showed similar gains (31). Most of the studies used self-report instruments to determine psychiatric diagnoses and substance use at baseline.
Two of the 14 studies also analyzed case management as a separate input from housing (7,29). Both found that more intensive case management had no effect on housing; neither study could support a hypothesis that higher levels of case management support would result in improved housing outcomes. Eight of the 14 studies analyzed well-being or other self-reported psychological states as an outcome in addition to housing or clinical outcomes (7,8,13,17,28,30,31,33). These outcomes, obtained by using a variety of self-report tools, also were mixed.
This analysis of interventions and outcomes found limitations in each of the 14 studies, such as attrition, selection and response bias, imprecise definitions and implementation of housing programs, and a lack of appropriate controls. These pitfalls limit the ability of the research to inform the policy goal of ending homelessness and demonstrate the need for further experimental research to inform funding and policy decisions. The body of research is unable to answer fundamental questions about what type of housing program works best for homeless individuals with mental illness.
Discussion
Sampling and Selection Bias
All but one (33) of the 14 studies had original sample sizes of more than 100 individuals, with a range from 49 to 3,811 and a mean of 613 (Table 1). Results were skewed by two quasi-experimental studies that used large, unmatched control groups (13,34). Excluding these studies, the range of participants was 411 and the mean was 217. All of the reviewed studies followed participants for at least one year. Four studies followed participants for up to three years (7,13,29,35), and one followed participants for five years (34).
| Study | Sample sizeb | Male | Age (years) | Psychotic disorders | Substance use | Experimental vs. control group outcome | ||
|---|---|---|---|---|---|---|---|---|
| Housing | Clinical | Case management | ||||||
| Lipton et al., 1988 (33) | 49 | Experimental, unknown; control, not reported | Experimental, unknown; control, not reported | Experimental, unknown; control, not reported | Experimental, unknown; control, not reported | More housed by end of study | Housing not associated with reduction in psychiatric symptoms | |
| Goldfinger et al., 1999 (35) | 118 | Experimental, 72%; control, 41% | M=38 | 62% | 50% | More housed by end of study | ||
| McHugo et al., 2004 (8) | 121 | Experimental, 8%; control, 83% | M=40 | 73% | 31%–54%c | More housed by end of study | Continuum housing associated with reduction in psychiatric symptoms; no change in substance use across groupsd | |
| Siegel et al., 2006 (28)e | 139 | Experimental, 57%–72%; control, 26%–54% | 35–48 | Experimental, unknown; control, not reported | Experimental, unknown; control, not reported | Similar number housed by end of study | ||
| Clark and Rich, 2003 (30) | 152 | Experimental, 52%; control, 20% | M=38 | Experimental, unknown; control, not reported | Experimental, unknown; control, not reported | Greater number with severe psychiatric symptoms housed; fewer with less severe psychiatric symptoms housed | ||
| Padgett et al., 2006 (37) | 173 | Experimental, 77%; control, 40% | M=42 | 90% | No difference in substance use | |||
| Greenwood et al., 2005 (31) | 197 | Experimental, 76%; control, 29% | M=42 | 54% | Experimental, unknown; control, not reported | Greater proportion of time housed | Similar reduction in psychiatric symptoms across groups | |
| Tsemberis et al., 2004 (32) | 206 | Experimental, 79%; control, 41% | 59% were between ages 31 and 50 | 53% | 90% | Greater proportion of time housed | Housing not associated with reduction in psychiatric symptoms or substance use | |
| Hurlburt et al., 1996 (29) | 361 | Experimental, 67%; control, 63% | 42% were between ages 30 and 39 | 55% | 53% | More living in independent housing, but experimental condition did not reduce homelessness | Case management not associated with positive housing outcome | |
| Gilmer et al., 2010 (17) | 363 | Experimental, 56%; control, 25% | M=44 | 60% | Experimental, unknown; control, not reported | Fewer days homeless and more days housed | ||
| Rosenheck et al., 2003 (7) | 460 | Experimental, 96%; control, not reported | M=42 | Experimental, unknown; control, not reported | 50% | More days housed | Housing not associated with reduction in psychiatric symptoms | Case management not associated with positive housing outcome |
| Cheng et al., 2007 (36) | 460 | Experimental, 96%; control, not reported | M=42 | Experimental, unknown; control, not reported | 50% | More days housed | Housing associated with reduction in substance use | |
| Tsemberis and Eisenberg, 2000 (34)f | 1,841 | Experimental, 67%; control, 73% | M=41 | Experimental, 60%; control, 46% | Experimental, 58%; control, 49% | More housed by end of study | ||
| Tsemberis, 1999 (13)g | 3,950 | Experimental, 74%; control, 66% | Experimental, 58% were between ages 30 and 49; control, 69% were between ages 30 and 49 | Experimental, 59%; control, 55% | Experimental, 62%; control, 49% | More housed by end of study | ||
a
Demographic data reflect baseline characteristics.
c
Aggregates four groups of data, by group condition and by alcohol and illicit drug use
d
The study did not hypothesize an outcome but compared two experimental groups, one adhering more to an independent housing model and the other to the continuum of care model.
e
To mimic randomization, the study divided participant data into strata on the basis of the probability that each individual would need a higher intensity of supported housing. The study reported demographic information by strata, resulting in a range of values for each variable.
f
Experimental group, N=241; nonrandomized, unmatched comparison group, N=1,600
g
Experimental group, N=139; nonrandomized, unmatched comparison group, N=3,811
Study participants were mostly male, with mean ages in the late 30s to mid-40s. Characteristics such as race-ethnicity, psychiatric diagnosis, and substance use differed across studies, although a majority of participants in most studies had psychotic disorders and a co-occurring substance use disorder. All study participants were homeless or at immediate risk of homelessness and had a serious mental illness, such as major depression, bipolar disorder, or schizophrenia, at baseline. Each study used different criteria for homelessness to establish eligibility, ranging from self-reported information of less than one month spent homeless to many months. Studies either used clinical records or interviewed participants with diagnostic tools to establish mental illness.
Thirteen of the 14 studies used a screening process to exclude participants from the study (7,8,13,28–37), although most contained few details on the process. The two studies that used Section 8 vouchers to house participants screened out individuals with histories of violence and criminal records (7,29), while another excluded those believed to be a danger to self or others (35). Two reported the number of individuals screened for eligibility relative to participants who gave informed consent (7 Three reported eligibility criteria for experimental-housing participants, with less detail on control groups (31,32,37). One study claimed not to screen out any participants from the study sample (17).
The studies obtained research participants from a mix of referral sources, threatening the external validity of housing and clinical outcomes. Some participants were recruited from a single source or type of source, such as one hospital, multiple hospitals, and multiple shelters, or from a combination of sources, including hospitals, shelters, and streets. Some studies relied on social services agencies to recruit participants who were not already receiving services, and other studies recruited clients or outreach workers aligned with existing programs. Such procedures may have positively influenced outcomes if referring sources referred individuals with less severe disabilities. Individuals discharged from hospitals and stabilized on medication, as well as those receiving services in shelters, may have better outcomes in housing than those placed directly from the streets (33,35). Two studies reported directing service providers to refer individuals who were capable of participating in housing programs or who were willing participants (29,32).
Attrition
Researchers have used various methods of calculating and reporting attrition, making comparisons difficult (38). The 12 primary studies, excluding the secondary analyses (36,37) reported attrition in a variety of ways (7,8,13,17,28–35). Seven studies reported attrition as the percentage of participants housed at the end of the study period (8,13,28,29,33–35), two reported attrition as the percentage of participants completing follow-up interviews (7,30), two reported attrition as the proportion of time spent homeless (31,32), and one reported days spent homeless (17).
Table 2 includes the percentage data from the nine studies that either analyzed the percentage of participants housed at the end of the study period or those completing follow-up interviews (7,8,13,28–30,33–35), excluding two studies that reported attrition as the proportion of time spent housed versus homeless (31,32) and excluding the secondary analyses (36,37). Results from one study were averaged with results from a secondary analysis that reported original outcomes that were slightly different from the outcomes of the primary study (29). Attrition rates were averaged for a third study in which attrition rates were reported as a range of percentage values (28). For a study with an experimental group and two control groups, data from the group that received case management only, which received more intensive services than the “standard care” group, were included (7).
| Attrition rate and study | Study period | Experimental group or group hypothesized to outperform comparison group (%)b | Control group or group hypothesized to underperform experimental group (%)c | Average attrition across experimental and control groups |
|---|---|---|---|---|
| 0%–29% (medium or high statistical power) | ||||
| McHugo et al., 2004 (8) | 18 months | 15 | 32 | 24 |
| Goldfinger et al., 1999 (35) | 3 years | 23 | 24 | 24 |
| Tsemberis, 1999 (13) | 3 years | 16 | 40 | 28 |
| 30%–49% (low or medium statistical power) | ||||
| Tsemberis and Eisenberg, 2000 (34) | 5 years | 12 | 53 | 33 |
| Siegel et al., 2006 (28) | 18 months | 28 | 46 | 37 |
| Rosenheck et al., 2003 (7) | 3 years | 23 | 52 | 38 |
| Clark and Rich, 2003 (30) | 1 year | 24 | 64 | 44 |
| ≥50% (very low statistical power) | ||||
| Lipton et al., 1988 (33) | 1 year | 31 | 70 | 51 |
| Hurlburt et al., 1996 (29) | 3 years | 42 | 70 | 56 |
a
Attrition was measured either as the percentage of participants who were housed by the end of the study (8,13,28,29,33–35) or as the percentage who completed follow-up interviews (7,30). Studies that measured homelessness as the proportion of time spent homeless (31,32) or days spent homeless (17) are not included in this table.
b
Mean=24%
c
Mean=50%
Six of the nine studies had mean attrition rates of more than 30% across experimental and control groups (7,28–30,33,34). These rates are similar to previously reported attrition rates involving homelessness research (30), but they are large enough to threaten statistical power. Attrition is common in longitudinal research on homeless individuals with mental illness (39). High attrition rates lower statistical power by reducing sample sizes and threatening internal and external validity (38). Large numbers of dropouts make it more difficult to attribute outcomes to the intervention and raise the question of whether differences are attributable to underlying differences between groups, such as the relative severity of mental or general medical illness.
All of the studies reporting attrition included dropouts in their statistical calculations of attrition rates and housing outcomes. Most of the studies used various statistical methods to correct for potential bias caused by attrition, including inputting missing data on the basis of assumed values, weighting adjustments for nonrespondents, and analyzing relationships between baseline scores and participant characteristics for those who stayed in the study and those who dropped out. Such methods can smooth out data points, which can suggest a lack of potential bias, such as a lack of association between dropouts and high levels of psychiatric symptoms that might indicate that attrition was not influenced by mental illness. Statistical corrections rely on untestable assumptions about chance events that if wrong could lead to inaccurate or misleading conclusions (40). Such methods do not protect against the potential risk that dropouts did not exit at random, were doing less well than their participating counterparts, or exited because of some program feature.
Only four of the 12 original studies reported the whereabouts of study participants during the program, at the end of the study period, or thereafter (17,31,33,35). None of the studies had specific explanations for why participants dropped out of the study or what precipitated their exit from housing, although some reported dropouts as “lost,” refusing follow-up interviews, leaving against staff advice, or not wanting to continue the program. Lack of detail on the reasons for attrition or unavailability for interviews is unsatisfactory for empirical research (38).
Response Bias
The studies determined housing status by interviewing study participants, although some of the studies attempted confirmation with management information systems or visits by case managers or research staff. Self-report interviews, especially of transient individuals with mental illness and substance use disorders, are subject to response error and misunderstanding even when formal tools are used (41).
Five of the 14 studies used a tested assessment tool developed by the Dartmouth Psychiatric Research Center called the Residential Time-Line Follow-Back Inventory to determine homelessness and residential status, although most of the reviewed studies used a variety of instruments (8,28,30–32). All but one (17) of the 14 studies reported the use of various self-report tools to determine mental health status or substance use. Studies involving homeless individuals with both mental illness and substance use disorders have shown a high rate of discrepancy between self-reported information on clinical status and observed data (41,42). In the absence of toxicology tests to determine substance use, the reliability of self-report information is questionable given the presence of mental illness and the risk of recall error, refusal to disclose information, intentional misrepresentation, or the provision of socially acceptable responses in the presence of authority figures.
Five of the 12 primary studies reported paying study participants for completing interviews about housing or mental health status (7,29–31,35). Although research compensation is common, paying individuals for participation may have biased results by encouraging socially acceptable answers or increasing greater participation in housing programs than would otherwise have been the case. Research reactivity may also have biased outcomes. The informed consent process prevents participants from being blinded to study conditions and, by extension, to broader goals and policy implications.
The reviewed studies may have been subject to demonstration effects because self-aware study participants may have performed better or responded more favorably to interview questions. Only one of the studies explicitly addressed this concern (35). The federal funds that financed many of the reviewed studies may also have contributed to more robust services and more motivated workers than would exist under nonresearch conditions.
Design and Implementation of Housing and Support Services
The reviewed studies defined, designed, and implemented supportive housing in a variety of ways. Most of the studies did not specifically refer to their experimental housing conditions as “supportive housing,” instead using a variety of terms, such as Housing First (13,31,32,34,37), comprehensive housing (30), evolving consumer households (35), integrated housing (8), and full-service partnerships (17).
Twelve of the 14 studies lacked detail or offered vague descriptions of experimental or control conditions (7,8,13,17,29–34,36,37); only two of the 14 studies reported sufficient detail to enable comparative analysis (28,35). All but one (33) of the 14 studies used regression analyses or other statistical methods to analyze associations between variables that may have affected housing or clinical outcomes. None of the studies could pinpoint what drove positive outcomes, however. Imprecise definitions and implementation of housing programs limit the usefulness of statistical analysis and threaten internal and external validity because individual variables, such as substance use, can become subject to influence by many other mediating variables that are not controlled for or excluded in experimental conditions.
Only three of the 14 studies reported using a housing fidelity assessment tool to test whether the housing intervention was faithful to theoretical standards (8,28,30). Few studies on supported housing have incorporated fidelity assessments, which are important in determining whether interventions are implemented as planned and whether outcomes can be attributed to the interventions as implemented (10,23). Research on supportive housing has suffered from vague housing models and lack of fidelity to these models (10). For example, the Pathways studies described the principles, policies, and procedures of their Housing First program but did not include a quantitative assessment of whether the model was consistently implemented (13,31,32,34,37). In studies that incorporated fidelity instruments, ratings showed that experimental programs were mostly faithful to their program models, strengthening internal validity. External validity was limited, however, because conceptions and implementation of housing varied widely across studies.
All of the studies included case management as part of their experimental housing programs, reflecting a long-standing view that case management is essential to helping individuals with mental illness live stably in the community (43–45). The studies generally included more information on the case management components of their programs, such as staffing ratios and time spent with clients, than on the housing settings. They fell short in terms of detailing how case management was implemented across study conditions and especially in describing differences in the quantity, variety, intensity, and quality of services in control versus experimental groups.
Three of the 12 primary studies contained information on case management staffing ratios (7,8,29), although only one of the three included ratios for both experimental and control groups (29). The three studies used various staffing ratios to quantify higher-intensity case management services, making comparisons of care standards difficult. Case management programs were not otherwise quantified.
The studies defined and implemented case management in a variety of ways. Seven studies used ACT teams (13,17,28,31,32,34,37) and others used services that included “intensive case management” (7,8,36), “intensive clinical case management” (35), and “comprehensive case management” alongside “traditional case management” (29). Two studies reported applying fidelity assessment tools that were either homegrown (30) or based on the Dartmouth Assertive Community Treatment Fidelity Scale (8); one study employed a “full fidelity” assertive community treatment team that met at least monthly with Housing First participants (17).
Six of the 12 primary studies stated how frequently case managers met with participants, although confirmation methods were unclear. Visits ranged from at least one per week (7,28,35) to at least two times per month (13,31,34). Five of the 14 studies analyzed the degree to which participants used services, and the results varied across studies as experimental groups used more or less than control groups (7,8,17,32,37).
How programs handled drug and alcohol use among participants also was unclear because only two studies elaborated on crisis response procedures (13,28). One Pathways study explained that tenants who relapsed did not lose housing and that case managers intensified their work with tenants to manage the problem (13). In the other study, supportive housing tenants were prescreened for evidence of six months of clean and sober behavior and could be asked to leave if they were disruptive (28). Lack of information raises questions about how Housing First programs are implemented, especially given rules prohibiting drug use in federally subsidized housing.
Randomization and Control Group Design
None of the reviewed studies included a no-treatment control group that lacked housing or other forms of assistance. Eight of the 14 studies compared two types of housing programs with each other, placing participants in some form of guaranteed housing (8,13,28,31,32,34,35,37); these studies suffered from imprecise descriptions of the housing interventions. Without control groups that lacked a housing component, it is difficult to estimate the degree to which housing contributed to reductions in homelessness or psychiatric symptoms.
Three studies contained an experimental condition in which priority access to Section 8 vouchers, but not housing itself, was guaranteed (7,29,36). These studies contained multiple control conditions that varied the intensity of case management assistance but did not guarantee housing. The three remaining studies compared a program that offered guaranteed housing with a condition in which participants were not offered guaranteed housing; instead of guaranteed housing participants were offered “active outreach and engagement, some onsite counseling, medication and medication management, assistance obtaining housing and linkages to other psychosocial services” (30), “routine discharge planning” from a hospital (33), or outpatient public mental health services (17).
The six studies that did not guarantee housing for all participants lacked detail on standard care control conditions (7,17,29,30,33,36). Outcome relevance is limited without more information about what “usual care” entails and whether it resulted in housing or temporary shelter for participants or no shelter at all. Ambiguous distinctions between experimental and comparison groups in housing and case management assistance limit overall credibility.
Five of the 14 studies were quasi-experimental in that they did not randomly assign participants into experimental and control conditions (13,17,28,30,34). Quasi-experimental studies are problematic in that outcomes may be result from differences between groups and unmeasured variables rather than to the interventions. “Readiness to change,” for example, could be one of several confounding variables influencing outcomes (17). Four of the five studies had experimental and comparison groups that were significantly different in several categories including gender, ethnicity, substance use, clinical diagnosis, and history of homelessness (13,28,30,34). Three of the five quasi-experimental studies used propensity scoring to mimic randomization and minimize bias (17,28,30). One assigned propensity scores to participants to match a control group against an experimental group along demographic indicators, excluding an unknown number of participants (17). None of the studies reported selecting research participants at random.
One Pathways study attempted to minimize bias by establishing a comparison group of participants referred from sources similar to the experimental Housing First group (34). Another Pathways study did not report taking such steps, although the author stated that bias was minimized because the experimental group was more disabled than the control group at baseline (13).
Conclusions
This review of the best studies on permanent supportive housing identified a small base of research with limited usefulness for decision makers seeking empirical evidence to justify policy choices. The research cannot yet pinpoint which factors drive positive housing and clinical outcomes. Research problems involving attrition, lack of detail on housing conditions and supports, selection bias, and lack of standardized program models and definitions limit internal validity, the ability to generalize findings, and efforts to replicate research conditions. With the possible exception of the Pathways studies, the reviewed studies are unique to their environments.
Supportive housing research relies on external and limited funding that may lead to program modifications (10). Many supportive housing studies do not employ randomization because of ethical concerns about assigning vulnerable individuals to control groups that withhold services (8,24). The At Home/Chez Soi demonstration project is the world’s largest Housing First randomized controlled trial, involving approximately 2,500 participants in five Canadian cities; it reported significant opposition to randomization, among other challenges (46–49). Ongoing results from this study are expected to make a significant contribution to the literature. Given the scope of services and participant needs subjected to study, including multiple degrees of disability across a broad demographic sample, results from the research may address many of the concerns raised in this article. Results should be addressed in the context of Canadian policy priorities and a more generous safety net than exists in the United States. At present, the best research—lacking a verifiable standard for the quality of housing and supports against which to test program variations—fails to detail the specific therapeutic benefits of supported housing beyond having a roof overhead.
Problems defining and conceptualizing variables may reflect overly broad applications of services such as case management, a treatment concept “used so broadly as to have no specific meaning at all” (50). The two studies measuring case management as an input independent of housing found that the more service-intensive condition did not influence housing outcomes, raising questions for future research about the therapeutic benefits of case management (7,29). Moreover, the studies in this review did not control for placebo affects arising from expected benefits. Assertive community treatment teams and intensive case management models have been modified over time, with no single standard of care for homeless individuals with mental illness (51). Casework is subject to variation with regard to house rules, crisis management, and treatment participation. Services may vary despite attempts to standardize them across experimental and control groups (7). Case managers are not disinterested observers and relate to clients on the basis of personality type, moral imperative, and institutional policy.
Recent fidelity models have identified similar features, such as immediate placement in housing and the lack of a mandate for treatment participation, as key factors distinguishing the Housing First formula from other programs (52–54). Faithful implementation remains in doubt, especially in light of provider preferences for treatment mandates and sober living programs (54). A qualitative study of permanent housing programs in California found significant variation in fidelity to the Housing First model, driven largely by the priorities and values of program directors (55). Future research could be strengthened by use of fidelity models that encompass every stage of the treatment process, from outreach and screening to follow-up care, as well as a range of supportive housing types with various degrees of psychiatric and case management involvement. Unraveling the effects of housing and support services presents a methodological challenge likely requiring multiple experimental and control groups (25).
In contrast to the reviewed research, recent qualitative studies have documented in detail how Housing First programs are implemented in practice, such as particular casework approaches to assisting substance users after housing placement (6,56–60). These studies generally conclude that “harm reduction” policies incorporating flexible housing rules improve housing stability. In the best available experimental research, fidelity to the supportive housing model remains a concern, as well as a challenge to the implementation and evaluation of standardized treatment methods.
Most of the reviewed studies did not analyze the effect of housing on mental health outcomes, whereas those that did were inconclusive. The chronic yet fluctuating nature of mental illness makes clinical outcomes difficult to interpret, because study participants may experience “regression to the mean” (8). More exploration of how housing and casework affect mental health is needed to analyze program effectiveness.
As service providers face a HUD mandate to end chronic homelessness by 2015, questions persist about the type of housing this population needs. The economic justification needed to fulfill society’s moral obligation to house homeless individuals with mental illness should be built on rigorous experimental research, which is expensive and time consuming but necessary to answer fundamental questions about efficacy.
Footnote
The opinions and conclusions in this article are solely those of the author and not necessarily those of her employer.
References
1.
Newman S, Goldman H: Housing policy for persons with severe mental illness. Policy Studies Journal 37:299–324, 2009
2.
The State of Homelessness in America. Washington, DC, National Alliance to End Homelessness, 2014
3.
DeNavas-Walt C, Proctor BD: Income and Poverty in the United States: 2013. Current Population Reports, P60-24. Washington, DC, US Census Bureau, 2014
4.
Montgomery AE, Metraux S, Culhane, D: Rethinking homelessness prevention among persons with serious mental illness. Social Issues and Policy Review 7:58-82, 2013
5.
Fazel S, Khosla V, Doll H, et al: The prevalence of mental disorders among the homeless in western countries: systematic review and meta-regression analysis. PLoS Medicine 5:e225, 2008
6.
Pearson CL, Locke G, McDonald WR: The Applicability of Housing First Models to Homeless Persons With Serious Mental Illness. Washington, DC, US Department of Housing and Urban Development Office of Policy Development and Research, 2007
7.
Rosenheck R, Kasprow W, Frisman L, et al: Cost-effectiveness of supported housing for homeless persons with mental illness. Archives of General Psychiatry 60:940–951, 2003
8.
McHugo GJ, Bebout RR, Harris M, et al: A randomized controlled trial of integrated versus parallel housing services for homeless adults with severe mental illness. Schizophrenia Bulletin 30:969–982, 2004
9.
Fakhoury WKH, Murray A, Shepherd G, et al: Research in supported housing. Social Psychiatry and Psychiatric Epidemiology 37:301–315, 2002
10.
Tabol C, Drebing C, Rosenheck R: Studies of “supported” and “supportive” housing: a comprehensive review of model descriptions and measurement. Evaluation and Program Planning 33:446–456, 2010
11.
Opening Doors: Federal Strategic Plan to Prevent and End Homelessness. Update 2013. Washington, DC, US Interagency Council on Homelessness, 2013. Available at usich.gov/resources/uploads/asset_library/USICH_Annual_Update_2013.pdf
12.
Notice of Funding Availability (NOFA) for the Fiscal Years 2013 and 2014: Continuum of Care Program Competition. Washington, DC, US Department of Housing and Urban Development, 2013. Available at www.onecpd.info/resources/documents/FY2013-2014CoCProgramNOFA.pdf
13.
Tsemberis S: From streets to homes: an innovative approach to supported housing for homeless adults with psychiatric disabilities. Journal of Community Psychology 27:225–241, 1999
14.
Ridgway P, Zipple AM: The paradigm shift in residential services: from the linear continuum to supported housing approaches. Psychosocial Rehabilitation Journal 13:11–31, 1990
15.
HUD Reports Continued Decline in US Homelessness Since 2010. Washington, DC, US Department of Housing and Urban Development, 2013. Available at portal.hud.gov/hudportal/HUD?src=/press/press_releases_media_advisories/2013/HUDNo.13-173
16.
Stanhope V, Dunn K: The curious case of Housing First: the limits of evidence based policy. International Journal of Law and Psychiatry 34:275–282, 2011
17.
Gilmer TP, Stefancic A, Ettner SL, et al: Effect of full-service partnerships on homelessness, use and costs of mental health services, and quality of life among adults with serious mental illness. Archives of General Psychiatry 67:645–652, 2010
18.
Culhane DP, Metraux S, Hadley T: Public service reductions associated with placement of homeless persons with severe mental illness in supportive housing. Housing Policy Debate 13:107–163, 2002
19.
Larimer ME, Malone DK, Garner MD, et al: Health care and public service use and costs before and after provision of housing for chronically homeless persons with severe alcohol problems. JAMA 301:1349–1357, 2009
20.
Gilmer TP, Stefancic A, Tsemberis S, et al: Full-service partnerships among adults with serious mental illness in California: impact on utilization and costs. Psychiatric Services 65:1120–1125, 2014
21.
Fitzpatrick-Lewis D, Ganann R, Krishnaratne S, et al: Effectiveness of interventions to improve the health and housing status of homeless people: a rapid systematic review. BMC Public Health 11:638, 2011
22.
Kyle T, Dunn JR: Effects of housing circumstances on health, quality of life and healthcare use for people with severe mental illness: a review. Health and Social Care in the Community 16:1–15, 2008
23.
Nelson G, Aubry T, Lafrance A: A review of the literature on the effectiveness of housing and support, assertive community treatment, and intensive case management interventions for persons with mental illness who have been homeless. American Journal of Orthopsychiatry 77:350–361, 2007
24.
Rog DJ: The evidence on supported housing. Psychiatric Rehabilitation Journal 27:334–344, 2004
25.
Newman SJ: Housing attributes and serious mental illness: implications for research and practice. Psychiatric Services 52:1309–1317, 2001
26.
Rosenheck R: Cost-effectiveness of services for mentally ill homeless people: the application of research to policy and practice. American Journal of Psychiatry 157:1563–1570, 2000
27.
Rog DJ, Marshall T, Dougherty RH, et al: Permanent supportive housing: assessing the evidence. Psychiatric Services 65:287–294, 2014
28.
Siegel CE, Samuels J, Tang DI, et al: Tenant outcomes in supported housing and community residences in New York City. Psychiatric Services 57:982–991, 2006
29.
Hurlburt MS, Wood PA, Hough RL: Providing independent housing for the homeless mentally ill: a novel approach to evaluating long-term longitudinal housing patterns. Journal of Community Psychology 24:291–310, 1996
30.
Clark C, Rich AR: Outcomes of homeless adults with mental illness in a housing program and in case management only. Psychiatric Services 54:78–83, 2003
31.
Greenwood RM, Schaefer-McDaniel NJ, Winkel G, et al: Decreasing psychiatric symptoms by increasing choice in services for adults with histories of homelessness. American Journal of Community Psychology 36:223–238, 2005
32.
Tsemberis S, Gulcur L, Nakae M: Housing First, consumer choice, and harm reduction for homeless individuals with a dual diagnosis. American Journal of Public Health 94:651–656, 2004
33.
Lipton FR, Nutt S, Sabatini A: Housing the homeless mentally ill: a longitudinal study of a treatment approach. Hospital and Community Psychiatry 39:40–45, 1988
34.
Tsemberis S, Eisenberg RF: Pathways to Housing: supported housing for street-dwelling homeless individuals with psychiatric disabilities. Psychiatric Services 51:487–493, 2000
35.
Goldfinger SM, Schutt RK, Tolomiczenko GS, et al: Housing placement and subsequent days homeless among formerly homeless adults with mental illness. Psychiatric Services 50:674–679, 1999
36.
Cheng AL, Lin H, Kasprow W, et al: Impact of supported housing on clinical outcomes: analysis of a randomized trial using multiple imputation technique. Journal of Nervous and Mental Disease 195:83–88, 2007
37.
Padgett DK, Gulcur L, Tsemberis S: Housing First services for people who are homeless with co-occurring serious mental illness and substance abuse. Research on Social Work Practice 16:74–83, 2006
38.
Cohen EH, Mowbray CT, Bybee D, et al: Tracking and follow-up methods for research on homelessness. Evaluation Review 17:331–352, 1993
39.
Kalton G, Kasprzyk D: The treatment of missing survey data. Survey Methodology 12:1–16, 1996
40.
Ribisl KM, Walton MA, Mowbray CT et al: Minimizing participant attrition in panel studies through the use of effective retention and tracking strategies: review and recommendations. Evaluation and Program Planning 19: 1–25, 1996
41.
Calsyn RJ, Morse GA, Klinkenberg WD, et al: Reliability and validity of self-report data of homeless mentally ill individuals. Evaluation and Program Planning 20:47–54, 1997
42.
Goldfinger SM, Schutt RK, Seidman LJ, et al: Self-report and observer measures of substance abuse among homeless mentally ill persons in the cross-section and over time. Journal of Nervous and Mental Disease 184:667–672, 1996
43.
Stein LI, Test MA: Alternative to mental hospital treatment: I. conceptual model, treatment program, and clinical evaluation. Archives of General Psychiatry 37:392–397, 1980
44.
Bond GR, Witheridge TF, Dincin J, et al: Assertive community treatment for frequent users of psychiatric hospitals in a large city: a controlled study. American Journal of Community Psychology 18:865–891, 1990
45.
Bond GR, McGrew JH, Fekete DM: Assertive outreach for frequent users of psychiatric hospitals: a meta-analysis. Journal of Mental Health Administration 22:4–16, 1995
46.
Goering PN, Streiner DL, Adair C, et al: The At Home/Chez Soi trial protocol: a pragmatic, multi-site, randomised controlled trial of a Housing First intervention for homeless individuals with mental illness in five Canadian cities. BMJ Open 1:e000323, 2011
47.
Hwang SW, Stergiopoulos V, O’Campo P, et al: Ending homelessness among people with mental illness: the At Home/Chez Soi randomized trial of a Housing First intervention in Toronto. BMC Public Health 12:787, 2012
48.
Aubry T, Tsemberis S, Adair CE, et al: One-year outcomes of a randomized controlled trial of Housing First with ACT in five Canadian cities. Psychiatric Services (Epub ahead of print, February 2, 2015)
49.
Nelson G, Patterson M, Kirst M, et al: Life changes among homeless persons with mental illness: a longitudinal study of Housing First and usual treatment. Psychiatric Services (Epub ahead of print, February 17, 2015)
50.
Mechanic D: Correcting misconceptions in mental health policy: strategies for improved care of the seriously mentally ill. Milbank Quarterly 65:203–230, 1987
51.
Coldwell CM, Bender WS: The effectiveness of assertive community treatment for homeless populations with severe mental illness: a meta-analysis. American Journal of Psychiatry 164:393–399, 2007
52.
Tsemberis S: Housing First: The Pathways Model to End Homelessness for People With Mental Illness and Addiction. Center City, Minn, Hazelden, 2010
53.
Gilmer TP, Stefancic A, Sklar M, et al: Development and validation of a Housing First fidelity survey. Psychiatric Services 64:911–914, 2013
54.
Watson DP, Orwat J, Wagner DE, et al: The Housing First Model (HFM) Fidelity Index: designing and testing a tool for measuring integrity of housing programs that serve active substance abusers. Substance Abuse Treatment, Prevention, and Policy, 2013 (doi )
55.
Gilmer TP, Katz ML, Stefancic A, et al: Variation in the implementation of California’s Full Service Partnerships for persons with serious mental illness. Health Services Research 48:2245–2267, 2013
56.
Watson DP, Wagner DE, Rivers M: Understanding the critical ingredients for facilitating consumer change in Housing First programming: a case study approach. Journal of Behavioral Health Services and Research 40:169–179, 2013
57.
Tiderington E, Stanhope V, Henwood BF: A qualitative analysis of case managers’ use of harm reduction in practice. Journal of Substance Abuse Treatment 44:71–77, 2013
58.
Collins SE, Clifasefi SL, Dana EA, et al: Where harm reduction meets Housing First: exploring alcohol’s role in a project-based housing first setting. International Journal on Drug Policy 23:111–119, 2012
59.
Henwood BF, Stanhope V, Padgett DK: The role of housing: a comparison of front-line provider views in Housing First and traditional programs. Administration and Policy in Mental Health and Mental Health Services Research 38:77–85, 2011
60.
Henwood BF, Padgett DK, Tiderington E: Provider views of harm reduction versus abstinence policies within homeless services for dually diagnosed adults. Journal of Behavioral Health Services and Research 41:80–89, 2014
Information & Authors
Information
Published In
Cover: The Violoncellist, by Lilla Cabot Perry, 1906. Private collection.
History
Received: 1 July 2014
Revision received: 5 October 2014
Revision received: 26 November 2014
Accepted: 22 December 2014
Published online: 15 April 2015
Published in print: August 01, 2015
Authors
Funding Information
The author reports no financial relationships with commercial interests.
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).