Requiring Sobriety at Program Entry: Impact on Outcomes in Supported Transitional Housing for Homeless Veterans

Data for this study, which was approved by an institutional review board, were provided by the VA Northeast Program Evaluation Center (NEPEC) and consisted of records of individuals who were admitted into, and discharged from, the GPD program in 2003–2005. The GPD program provides grants to community providers to acquire housing facilities or renovate them (or both) for veterans and provides per diem funds to defray the cost of operations and supportive services. The program provides housing for a period up to two years and is designed as a transitional program leading to permanent housing. Community providers differ in terms of the eligibility requirements for admission and the mix of services provided. Homeless veterans typically enter the program from the street or a shelter but may move into GPD programs directly from a halfway house or other short-term housing situation. To examine program outcomes, we analyzed records for veterans admitted on or after the first day of federal fiscal year 2003 and discharged on or before the last day of fiscal year 2005.

The data set was derived from three sources of information. The first was Form X, a structured interview administered by program staff to veterans entering specialized VA homeless programs that captures sociodemographic, psychosocial, health, housing, and employment information, as well as staff diagnostic impressions based on interview findings and client presentation. The second source was Form D, which captures information (including reason for discharge, place of residence, and work status) at the point of program discharge. The third was the Facility Survey, which provides information about the community provider (such as eligibility requirements and number of housing units). Information from these sources is collected and maintained by NEPEC.

Several restrictions reduced the size of the data set. Because the range of admissions per veteran was large (from one to seven admissions), we used only first admissions for each veteran, resulting in a data set for 17,932 unique veterans who were primarily male (97%) with a mean±SD age of 48.4±8.0. To ensure accurate information from Form X, we required it to have been administered within ten days of program admission. This restriction reduced the data set to 10,188 records. To adequately test the research questions, we also required the veterans to have been homeless for more than 30 days before completion of Form X, to have spent at least 25 of the previous 30 days in a setting that did not prohibit or prevent alcohol or drug use (that is, veterans were not in a prison or hospital), to have been housed in a program that did not exclude veterans with any lifetime history of alcohol or drug abuse, and not to have been discharged from the housing program because of serious general health or mental health problems requiring transfer to acute treatment. Finally, in order to ensure the reliability of the determination of sobriety on admission, we restricted selection of records from programs requiring sobriety at admission (the sober condition) to those that mandated that clients have 14–90 days of sobriety immediately before admission. Programs without a sobriety requirement (the unrestricted condition) admitted clients without regard to alcohol or drug use before admission. The final data set consisted of 3,188 records, 1,250 from programs requiring sobriety at admission and 1,938 from programs not requiring sobriety.

Preliminary analyses examined program characteristics of providers. Descriptive statistics and group comparisons for data from these programs were then conducted on variables in the following domains: demographic characteristics, general medical and psychiatric history, work and financial support, and treatment outcomes. Chi square statistics were calculated to determine significant differences between sobriety and unrestricted conditions on categorical variables. Independent-sample t tests were used to examine differences on continuous variables. When Levene's test for homogeneity of variances was significant, the examination of mean differences was adjusted accordingly.

Because the sample was large, we anticipated that many analyses would produce statistically significant results. To guide meaningful interpretation of findings, we calculated effect sizes for all analyses. Cohen's d was computed to provide an effect size for t tests (5). Following Cohen's recommendations, we interpreted absolute values for d of .2, .5. and .8 as indicating small, medium, and large effect sizes, respectively. Absolute values for d that were less than .2 were designated as meaningless, regardless of levels of statistical significance. We calculated Cramér's phi (ϕ) to provide an effect size estimate for chi square analyses (6). Phi is a measure of association between two binary variables and is similar to the Pearson correlation coefficient in its interpretation (7). Again following Cohen (5), we interpreted absolute values of phi of .10, .30, and .50 as indicating small, medium, and large effect sizes, respectively. Absolute values of phi that were less than .1 were designated as meaningless, regardless of levels of statistical significance. Because we focused interpretation of results primarily on effect sizes, we did not correct p values for the number of analyses that were conducted.

Logistic regression was used to examine predictors of treatment outcomes, with program completion, recidivism for homelessness, housing status, and employment status as outcome variables. In these analyses, predictor variables (demographic variables and psychiatric history variables) and condition (sobriety requirement versus unrestricted) entered the analysis under the forward method of entry with the Wald statistic as a criterion. In order to estimate the amount of variance in outcomes explained by these variables, we calculated the McKelvey-Zavoina index per the recommendation of DeMaris (8). This index also serves as a measure of effect size, with an explanation of 1% of the variance in the dependent variable considered to be the lower boundary of a small meaningful effect (5). Separate logistic regression analyses were conducted for each outcome variable.

The characteristics of our sample and program did not allow us to replicate exactly the analyses of Edens and colleagues (3). They compared substance-abusing housing clients and abstaining housing clients who were being served by the same set of programs. In our analyses, participants in the programs with admission conditional on sobriety were by definition sober for at least two weeks, whereas most participants (60%) in the programs in the unrestricted condition had used alcohol or drugs in the 30 days prior to program admission. We did, however, examine whether there was an interaction of past alcohol or drug abuse (more than 30 days prior to program entry) with program type (sobriety requirement versus unrestricted) on outcomes. These analyses used the same logistic regression approach described above. All analyses were carried out in SPSS, version 18.0 (9).

The 1,938 veterans in the unrestricted condition received services from 59 providers that admitted from one to 181 veterans. The largest provider delivered services to 9% of the sample. The 1,250 veterans in the sobriety condition received services from 49 providers that admitted from one to 373 veterans. One housing provider that required sobriety offered services to more than 10% of the sample, raising the question of whether this provider was unduly influential in affecting results for persons in the sobriety requirement group. To test this possibility, we conducted chi square analyses to compare outcomes between veterans receiving services from this large site and veterans receiving services at all other sites in the study that required sobriety. The analyses revealed that only veterans receiving services at the large site were significantly (p<.01) more likely to complete the program (60%) than veterans at the other sites that required sobriety (50%) and were significantly (p<.01) more likely to have a residence on discharge (66%) than veterans at the other sites that required sobriety (55%). However, neither of these differences in proportions was large enough to meet the criterion for a small effect size, suggesting that there were no meaningful differences in outcomes between the larger site and other sites that required sobriety. We therefore conducted all subsequent analyses on data from veterans receiving services from all 49 sites that required sobriety. Table 1 provides descriptive statistics and the results of t tests and chi square analyses of comparisons between sites that required sobriety and those that did not.

Examination of demographic characteristics revealed no differences between conditions in age or in the proportions of married veterans or veterans in rural areas. The proportion of white clients was significantly but only slightly higher in the unrestricted programs (Table 1). Comparisons of clients in the two conditions revealed no differences in the proportions who had full-time employment in the previous three years or those who had worked at all in the 30 days before admission. There were also no differences in the proportions of clients receiving VA, non-VA, or other sources of financial support.

Numerous comparisons between clients in the two study conditions on measures of general health, substance use, and mental health were conducted. A significant difference of small effect size showed a higher number of reported medical problems (such as diabetes and hypertension) in the unrestricted group. However, no difference was found between groups in the proportion of clients responding to the question of whether they believed they had a serious medical problem, and clients in housing contingent on sobriety were significantly (but only slightly so) more likely to have used VA services in the past six months. Expected significant differences (given the criterion of a minimum of 14 days of abstinence for supported housing programs requiring sobriety) of small to moderate effect sizes were found in comparisons of measures tapping days of alcohol and drug abuse in the month before program admission. Analysis of proportions of clients reporting current substance dependence showed a small effect difference with more frequent use in the unrestricted condition. Even when significant, differences between conditions in other alcohol and drug use variables did not reach the level of a meaningful effect.

A significant difference of small effect size was found between conditions showing that a greater proportion of clients in the unrestricted condition reported experiencing serious depression in the 30 days before program admission (Table 1). A significantly larger percentage of clients in unrestricted programs than in those requiring sobriety also reported suicidal ideation in this same time frame, but the difference was questionably meaningful. There were no differences between clients in the two conditions in suicide attempts or in use of psychiatric medications in the 30 days before program admission

Staff members of homeless programs suspected high frequencies of diagnoses of alcohol abuse, drug abuse, and mood disorder among clients in both types of housing program (Table 1). These diagnostic impressions were significantly more common in the unrestricted group and were characterized by a small effect size. Clients in housing with a sobriety requirement were significantly more likely than the other clients to be suspected of a diagnosis of posttraumatic stress disorder (PTSD), but this difference was questionably meaningful.

A small and equivalent majority of clients in both groups finished their homeless programs; however, clients in sobriety-required group stayed in their programs significantly longer. This difference was substantial enough to qualify as an effect of medium size. A significant but small effect was found in favor of clients in the sobriety condition to have a residence on discharge. However, after program discharge, clients in the unrestricted group were significantly more likely to be employed. They were also more likely to reenter a homeless program during the time frame of the study. The effect sizes for these latter two findings call the importance of these findings into question, however.

We first examined the treatment outcome variables to determine their independence. Intercorrelations among the variables were small (r<.24 for all correlations), suggesting little overlap in potential predictive value. Table 2 presents the results of logistic regression analyses conducted to predict treatment outcome, with program completion, recidivism for homelessness, housing status, and employment status as outcome variables. Predictor variables included program type (sobriety required versus not required) and Form X variables included sociodemographic, psychosocial, health, housing, and employment information but excluded variables reflecting current alcohol or drug use or diagnostic impressions of current alcohol or drug abuse.

The regression analyses failed to show the sobriety requirement to be a significant predictor of any program outcome. There were several significant predictors of program completion, recidivism for homelessness, and homeless-on-discharge outcome measures, but none explained sufficient variance in the outcome measure to achieve a small effect size. The entire sets of significant predictors for these outcome measures accounted for less than 3% of the variance (Table 2). Two variables reflecting receipt of VA and non-VA financial support were found to be significant, meaningful predictors of employment on discharge. For both predictors, individuals receiving support were less likely to be employed on leaving the program.

Logistic regression analyses of the effect of past alcohol or drug abuse, program sobriety requirement, and their interaction on program completion, recidivism for homelessness, housing status, and employment status produced an inconsistent pattern of occasional significant results. For each analysis, however, the total contribution to variability in outcome for all three predictor variables was less than 1%.