Housing and Employment Outcomes for Mental Health Self-Direction Participants
Abstract
Objective:
In self-direction, participants control individual budgets, allocating service dollars according to needs and preferences within program parameters to meet self-defined recovery goals. Mental health self-direction is associated with enhanced wellness and recovery outcomes at lower or similar cost than traditional service arrangements. This study compared outcomes of housing independence and employment between individuals who participated in self-direction and those who did not.
Methods:
This quasi-experimental study involved administrative data from 271 self-directing participants. Using coarsened exact matching with observed demographic, diagnostic, and other characteristics, the authors constructed a comparison group of non–self-directing individuals (N=1,099). The likelihood of achieving positive outcomes between first and last assessments during the approximately four-year study period was compared for self-directing and non-self-directing individuals.
Results:
Self-directing participants were more likely than nonparticipants to increase days worked for pay or maintain days worked at 20 or more days in the past 30 days (number needed to treat [NNT]=18; small effect size) and maintain or attain independent housing (NNT=16; small effect size), when analyses controlled, to the extent possible, for observed individual characteristics.
Conclusions:
Based on data from the nation’s largest and longest-standing program of its kind, results suggest that mental health self-direction is associated with modest improvements or maintenance of positive outcomes in employment and housing independence. This research adds to the literature examining self-direction in the context of mental health and begins to fill the need for a greater understanding of self-direction’s relationship to outcomes of interest to service users and families, providers, and system administrators.
Self-direction involves control by participants with serious mental health conditions of a portion of funds typically spent on their mental health treatment. Self-directing participants allocate individual budgets in a manner of their choosing within a set of program parameters, selecting and purchasing goods and services to work toward their goals. Typically, a coach or broker supports participants, facilitating development of person-centered plans, helping track progress toward goals and objectives, and assisting participants with financial management (1–3). Self-direction purchases are broad ranging and may include clinical services and nontraditional expenditures, such as transportation, vision and dental care, and computers (1,4). Self-direction arrangements take different forms and are housed in and overseen by various entities, including mental health authorities, managed care entities, social service agencies, and advocacy organizations (5). At last count, approximately 700 individuals with serious mental health conditions participate in self-directed services in the United States (5), and mental health self-direction is expanding worldwide (6).
A 2014 systematic review examining 15 studies, including four conducted in the United States, concluded that although mental health self-direction is associated with increased quality of life and cost-effectiveness, extant research had significant methodological limitations. The authors called for more rigorous research to inform policy and practice (7). Since that review, Spaulding-Givens and Lacasse (2) published a descriptive study of Florida Self-Directed Care (FloridaSDC). In that study, participants had modest improvements in mental health–related functioning and spent a majority of days in the community as opposed to institutional settings. A recent qualitative study documented gains by self-directing participants in several domains, including vocational pursuits, independent housing, and community inclusion (1). Another study explored purchase types and concluded that self-direction enabled individuals to better address their needs through individualized strategies that support wellness and increase engagement in meaningful activities (4).
This study adds to this evidence base. Its purpose was to compare two functional outcomes, employment and independent housing, between individuals who did and did not participate in mental health self-direction. These domains reflect self-direction’s value base, which is rooted in principles of recovery, independence, and community inclusion. A recent Institute of Medicine report highlighted similar indicators as key social determinants of health (8), and both were identified as priority outcomes in the report of the President’s New Freedom Commission on Mental Health (9) and in the Substance Abuse and Mental Health Services Administration’s Description of a Good and Modern Addictions and Mental Health Service System (10). Service users, families, and administrators alike value these outcomes, which enhance quality of life and community inclusion and may reduce reliance on publicly funded systems. Using a quasi-experimental design with coarsened exact matching (11) and logistic regression, we estimated odds of change or maintenance of a positive outcome in employment and in housing independence among individuals who were and were not self-directing.
This study used data from FloridaSDC, the country’s oldest and largest mental health self-direction effort. There are two FloridaSDC programs in the state, which contracts mental health services through regional systems of care overseen by nonprofit managing entities. These voluntary programs enroll approximately 330 individuals and are funded through a combination of state and local funds. To be eligible for either FloridaSDC site, individuals must be at least 18 years old and designated as having a serious and persistent mental illness as determined by the state. Individuals must also have permanent residence in the program areas and rely on public funds to cover mental health services. If individuals are not enrolled in Supplemental Security Income, Social Security Disability Income, or veterans’ benefits, they must be in the process of applying for those benefits. FloridaSDC participants must be legally competent to make financial decisions and may not be enrolled in assertive community treatment. During the study period, SDC participants enrolled in public benefits received a yearly budget of approximately $1,900 per year, which could be used in addition to their existing insurance coverage. Uninsured individuals received an approximately $3,700 yearly budget, with half reserved for outpatient mental health treatment, such as psychiatric treatment and therapy. With support from a coach and within program policy guidelines, participants link purchases to specific recovery goals in their person-centered plans. Figure 1 depicts a breakdown of FloridaSDC spending by purchase category at one site (where such data were available). Most purchases were related to goods and services other than mental health treatment (some participants accessed mental health treatment by using Medicaid or other benefits, and these services are not included in Figure 1), from transportation and dentistry to housing and employment-related supports.
FIGURE 1. Self-direction spending by purchase category for program A, July 2010 to April 2015a
aThe figure includes the full prematch sample. A sensitivity analysis that included only program participants who were successfully matched to a comparison case was conducted, and no major differences were found. The figure reflects only goods and services purchased with the self-directed budget; not all services used during the study period are included.
Methods
The Brandeis University Institutional Review Board and the Florida Department of Children and Families’ human protections administrator reviewed and approved the study protocol.
Data and Variables
Data were obtained as limited, deidentified data sets from the two managing entities that oversee the FloridaSDC programs, with consent from the state. According to state regulations, managing entities must collect demographic information, service event data, and mental health outcomes data. Providers report demographic information to managing entities at admission and whenever that information changes, and mental health outcome assessments are reported at first assessment, quarterly thereafter, and at discharge.
The data sets included demographic and outcomes information for all adults with a serious and persistent mental illness designation. Information regarding self-direction budget purchases and service use details were unavailable for both programs; therefore, this information was not analyzed in this study. The data sets used in the study covered different periods because the managing entities assumed responsibility at different times and transferred the data at different times. The program A study period spanned 4.8 years, beginning July 1, 2010. The study period for program B covered three years, beginning July 1, 2012. Because of these disparate periods and because of demographic differences, the data sets were kept separate during the selection of matched comparison cases, and results are reported by site as well as in the aggregate.
Individuals enrolled in FloridaSDC were categorized as the intervention group, and individuals who never enrolled in FloridaSDC were categorized as the comparison group. Before data cleaning, the data set included 21,183 individuals, 403 of whom were SDC participants. Comparison group individuals who did not meet FloridaSDC eligibility criteria at any point during the study period (under age 18, enrolled in Florida assertive community treatment, or resided outside the geographic service area) were removed, as were individuals with only one assessment. After data cleaning and before matching, there were 403 FloridaSDC participants and 12,209 nonparticipants.
We created two binary dependent variables—employment and independent housing—by comparing measurements at first and last assessment for each individual. Outcome variables, selected on the basis of study aims, were constructed to capture both positive change and maintenance of a positive outcome over the study period. There were three available indicators for exploring employment: employment status, income from paid employment (past 30 days), and days worked for pay (past 30 days). We selected days worked for pay because it reflects work behavior (as opposed to income) most accurately and in greatest detail. Individuals with a positive employment outcome were those who either increased the number of days worked in the past 30 days at last assessment compared with first assessment or reported 20 or more days of work at both assessments. For housing independence, we constructed an outcome variable indicating transition from dependent housing (dependent living with others, group home, assisted living, supported housing, or hospital) or homelessness to living independently or maintenance of independent housing status at first and last assessments.
We defined a positive program effect as either a positive change or maintenance of an already good outcome to account for the dual purposes of FloridaSDC—that is, helping participants gain access to employment and independent living, as well as preventing them from losing the resources they already have. The preventive goal is important to capture because the target population is at continued risk of losing access to employment or independent living. Focusing only on participants who lacked access to these resources at first assessment would miss the effects of the program in preventing loss of resources. For example, if SDC participants are more likely than non-SDC participants to remain living independently from first to last assessments, this positive outcome would not be evident from examining changes in independent living status over time.
Matching and Analysis
We used coarsened exact matching to create a non-SDC comparison group (11,12). This straightforward method is designed to reduce imbalances between groups by using specified categorical variables. The matching algorithm (available as an SPSS add-on) constructs a stratum for each combination of covariates and matches each intervention participant to one or more nonintervention individuals sharing the same stratum. The algorithm then produces weights to account for postmatch group size differences (13). This method was chosen over other matching methods (for example, propensity scores) because of the high number of categorical variables in the data set and because it results in matched cases that share identical values (12).
Factors hypothesized to affect both the outcomes and participation in FloridaSDC were selected as matching variables (14). Variable choice was informed by reviewing prematch descriptive statistics, key informant interviews with individuals knowledgeable about self-direction, and information gleaned from previous FloridaSDC evaluations (1,2,5,15–17). We selected the following matching variables: age, high school completion, gender, ethnicity (program A), race (program B), schizophrenia diagnosis, substance use disorder diagnosis, marital status, county of residence, veteran status, limited English proficiency, arrests (ever arrested during the study period), activities of daily living (ever assessed as unable to perform activities of daily living during the study period), community tenure (ever spent one or more days out of the community during the study period), days between first and last assessments, and disability income receipt. We successfully matched 67% (N=271) of the SDC participants to 1,099 non-SDC participants. Postmatch weights were used in all subsequent analyses.
We fitted two logistic regression models to estimate the likelihood of achieving a positive outcome on each of the dependent variables, controlling for relevant covariates. Model specifications were guided by a priori theory and methodological considerations. To control for differences in outcome variables at the first assessment, we included the value of the outcome measure at the first assessment as a covariate in each model. The set of standard covariates was age, gender, race, ethnicity, high school completion, county of residence, program site, veteran status, marital status, physical disability, schizophrenia diagnosis, substance use disorder diagnosis, disability income receipt, days between first and last assessment, value of the outcome measure at first assessment, and any of the following during the study period: arrested, unable to perform activities of daily living, admitted to services as incompetent or involuntary, and spent one or more days out of the community.
Results
Descriptive statistics for the SDC and non-SDC groups before and after matching are presented in Table 1. Before matching, the groups were significantly different on nearly all measures. After matching, there were no statistically significant differences between the SDC and non-SDC groups.
| Characteristic | Before matching | After matching | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| SDC (N=403) | Non-SDC (N=12,309) | SDC (N=271) | Non-SDC (N=1,099) | |||||||
| N | % | N | % | pa | N | % | N | % | pa | |
| Demographic | ||||||||||
| Age (M±SD) | 52.15±10.61 | 44.86±14.46 | <.001 | 51.99±10.28 | 51.78±11.10 | .769 | ||||
| Female | 272 | 67 | 7,593 | 62 | .018 | 196 | 72 | 791 | 72 | .908 |
| White race | 283 | 70 | 9,541 | 78 | <.001 | 200 | 74 | 839 | 76 | .381 |
| Black race | 94 | 23 | 1,971 | 16 | <.001 | 56 | 21 | 215 | 20 | .684 |
| Hispanic ethnicity | 20 | 5 | 1,478 | 12 | <.001 | 8 | 3 | 28 | 3 | .709 |
| Completed high school at first assessment | 319 | 79 | 7,812 | 64 | <.001 | 222 | 82 | 899 | 82 | .964 |
| Married at any assessment | 46 | 11 | 3,134 | 25 | <.001 | 22 | 8 | 104 | 9 | .493 |
| Employed full- or part-time at first assessment | 45 | 11 | 2,121 | 17 | .001 | 29 | 11 | 135 | 12 | .469 |
| Resides in county with the largest share of SDC participants | 293 | 73 | 5,170 | 42 | <.001 | 190 | 70 | 733 | 67 | .283 |
| Veteran of U.S. armed forces | 43 | 11 | 488 | 4 | <.001 | 2 | 1 | 8 | 1 | .986 |
| English severely limited at any assessment | 5 | 1 | 534 | 4 | .002 | 0 | — | 0 | — | — |
| Ever arrested | 7 | 2 | 672 | 5 | .001 | 2 | 1 | 10 | 1 | .786 |
| Diagnostic | ||||||||||
| Ever diagnosed as having a substance use disorder | 78 | 19 | 3,996 | 32 | <.001 | 42 | 15 | 201 | 18 | .281 |
| Ever diagnosed as having schizophrenia | 121 | 30 | 2,644 | 21 | <.001 | 71 | 26 | 279 | 25 | .784 |
| Ever assessed as unable to perform activities of daily living | 60 | 15 | 3,500 | 28 | <.001 | 39 | 14 | 179 | 16 | .445 |
| Ever assessed as having a physical disability | 59 | 15 | 1,280 | 10 | .006 | 39 | 14 | 152 | 14 | .811 |
| Benefits and services | ||||||||||
| Days in system (M±SD) | 1,045.15±401.18 | 881.52±498.87 | <.001 | 1,025.14±404.38 | 1,042.77±519.65 | .546 | ||||
| Received psychiatric disability income at last assessment | 297 | 74 | 3,848 | 32 | <.001 | 197 | 73 | 803 | 73 | .902 |
| Ever spent one or more days out of the community | 111 | 28 | 4,151 | 34 | .010 | 64 | 24 | 296 | 27 | .266 |
| Ever admitted to services as incompetent or involuntary | 98 | 24 | 2,788 | 23 | .432 | 64 | 24 | 270 | 25 | .744 |
a
Tests for significance were two-tailed, independent-samples t tests for continuous variables and two-tailed Pearson's chi-square tests with asymptotic p value reported for dichotomous variables. The dichotomous variables were coded 1 for clients who displayed the indicated t characteristic and 0 otherwise.
Table 2 presents unadjusted mean values of outcome variables at first and last assessment for SDC and non-SDC groups. The mean differences depicted in Table 2 do not capture maintaining a desirable outcome over the study period. To capture maintenance in addition to positive gains, we used the outcome variables described above rather than differences in means or proportions with improved outcomes for the multivariate analysis. Notably, SDC participants’ first assessments did not constitute a true baseline because 58% (N=158) of the SDC group were enrolled in FloridaSDC prior to the beginning of the study period. Some of the observed first-assessment differences may be due to having received FloridaSDC prior to the first observation. The regression models adjusted for this nonequivalence by estimating odds ratios (ORs) while controlling for the first-assessment values of outcome measures.
| SDCa | Non-SDCb | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| First assessment | Last assessment | First assessment | Last assessment | |||||||
| Variable | N | % | N | % | Unadjusted difference between first and last | N | % | N | % | Unadjusted difference between first and last |
| Program Ac | ||||||||||
| Days worked for pay in past 30 days (M±SD) | 2.52±6.44 | 4.81±7.74 | 2.29 | 3.22±7.49 | 5.30±9.37 | 2.08 | ||||
| Lives independently alone or with others | 107 | 86 | 112 | 90 | 4 | 448 | 72 | 506 | 81 | 9 |
| Program Bd | ||||||||||
| Days worked for pay in past 30 days (M±SD) | 1.33±3.95 | 2.14±5.76 | .81 | 1.53±5.52 | 2.44±6.90 | .91 | ||||
| Lives independently alone or with others | 141 | 97 | 139 | 96 | –1 | 328 | 73 | 361 | 81 | 7 |
| Programs A and B | ||||||||||
| Days worked for pay in past 30 days (M±SD) | 1.88±5.26 | 3.37±6.86 | 1.49 | 2.51±6.78 | 4.13±8.55 | 1.62 | ||||
| Lives independently alone or with others | 248 | 92 | 252 | 93 | 1 | 785 | 72 | 875 | 81 | 8.9 |
a
Program A, N=124; program B, N=147
b
Program A, N=642, program B, N=457
c
Data were missing for the following variables in program A: days worked for pay in past 30 days for the non-SDC group (N=14) and lives independently alone or with others for the non-SDC group (N=16).
d
Data were missing for the following variables in program B: lives independently alone or with others for the SDC group (N=2) and the non-SDC group (N=9).
We used logistic regression to produce adjusted ORs of SDC membership predicting the likelihood of a positive outcome, controlling for observed first-assessment nonequivalences (Table 3). The logistic regression results indicated that, when covariates were held constant, SDC participants in both programs were significantly more likely than the non-SDC group to experience a positive outcome in days worked for pay (OR=1.73, p≤.01). SDC participants had more than twice the odds (OR=2.04, p<.05) of maintaining or attaining independent housing, compared with the non-SDC group, when the analysis controlled for covariates. We calculated two indicators of effect size: Cohen’s h and number needed to treat (NNT). Cohen’s h was .18 for employment and .21 for independent living. To achieve a positive employment outcome for one participant, 18 participants need to be enrolled in SDC for three years (the sample mean for length of enrollment). The corresponding NNT for a positive independent living outcome was 16 participants. As discussed above, these numbers reflect both improvements and maintenance of positive outcomes and were adjusted for all model covariates, including outcome values at first assessment. They indicate small effect sizes, with a larger effect size on independent living than on employment.
| Variable | Model N | ORb | 95% CI | df | p | –2 log likelihood | NNTc | Cohen's h |
|---|---|---|---|---|---|---|---|---|
| Program A | ||||||||
| Increasing days worked in past 30 days or maintaining days worked at ≥20 in past 30 days | 746 | 1.71 | 1.06–2.75 | 19 | <.05 | 697.658 | 10 | .23 |
| Attaining or maintaining independent residential status | 757 | 1.80 | .87–3.74 | 19 | .113 | 519.993 | 29 | .14 |
| Program B | ||||||||
| Increasing days worked in past 30 days or maintaining days worked at ≥20 in past 30 days | 604 | 1.86 | .99–3.49 | 18 | .054 | 316.331 | 21 | .17 |
| Attaining or maintaining independent residential status | 598 | 2.96 | 1.09–8.03 | 18 | <.05 | 301.094 | 15 | .26 |
| Programs A and B | ||||||||
| Increasing days worked in past 30 days or maintaining days worked at ≥20 in past 30 days | 1,350 | 1.73 | 1.20–2.50 | 20 | <.01 | 1,087.627 | 18 | .18 |
| Attaining or maintaining independent residential status | 1,355 | 2.04 | 1.17–3.58 | 20 | <.05 | 898.984 | 16 | .21 |
a
The study period was July 1, 2010, to April 30, 2015, for program A and July 1, 2012, to June 22, 2015, for program B. All models included a standard set of covariates: age, gender, race, ethnicity, high school completion, marital status, program site (for aggregated model only), county of residence, veteran status, physical disability, schizophrenia diagnosis, substance use disorder diagnosis, receipt of disability income, days between first and last assessment, ever arrested, ever assessed as unable to perform activities of daily living, ever admitted to services as incompetent or involuntary, ever spent one or more days out of the community, and the value of the outcomes measure at first assessment.
b
Reference: outcome not achieved
c
Number needed to treat
Discussion
Using several years of administrative data, we found evidence suggesting positive effects of self-direction on both outcomes of interest: employment and independent housing. Although these results are promising, findings should be regarded with caution because of study limitations. Although matching eliminated all observed differences between SDC and comparison groups, unobserved group differences likely remained. As is typical of quasi-experimental designs, matching and statistical controls minimized but did not guarantee the elimination of case mix issues, so the results should be regarded as preliminary. Furthermore, only 67% of SDC participants were successfully matched, and approximately one-third of SDC participants were thus not included in the analyses. However, in preliminary analyses we used only 10 variables to define the matching algorithm (versus the 16 variables in the final analysis); while approximately 90% of the sample was successfully matched, the balancing between the groups was inadequate. Therefore, we chose to sacrifice the number of matches for quality of matches, and the overall results were robust to these varying specifications. Future studies should collect more detailed background information in order to account for a broader range of potential nonequivalence factors. However, it is worth noting that complete intergroup equivalence is notoriously hard to establish in studies of socially complex outcomes, even with randomized controlled designs (18).
Second, participants’ first assessments did not constitute a true baseline because a majority of the FloridaSDC group were enrolled prior to the beginning of the study period. Reported group differences in outcomes between first and last assessments are net of first-assessment differences, possibly masking some intervention effects that might occur before first assessment. More accurate estimates of intervention effects would have been possible with a design utilizing true baseline data.
To assess the degree to which the lack of a true baseline affected our results, we repeated the analysis limiting the SDC group to the 158 participants who enrolled during the study period and thus had a true baseline. For all models, the results were in a direction similar to that in the full sample. SDC enrollment remained a statistically significant predictor of the employment outcome in the combined sample (OR=2.22, p<.01), whereas its effect on independent living did not reach significance (OR=1.87, p=.165). The difference in significance level is to be expected given the reduction in sample size, and the sensitivity analysis suggested that the program effects reported above were not unduly biased by the lack of a true baseline.
Because the study was limited to available administrative data, potential confounders—including information about SDC-purchased goods and services and implementation-related factors, such as informal screening practices—were not available in the data. Furthermore, the data were collected by providers for billing and reporting purposes and are thus subject to possible unmeasured biases. Although it would have been desirable to investigate self-direction’s relationship to service use, detailed service use information was not available.
Despite its limitations, this study had unique and important strengths. The outcomes have import for publicly funded mental health systems, and possible improvement in the areas of employment and independent housing suggests that persons are enhancing their independence and self-sufficiency. These findings are supported by a recent qualitative analysis of in-depth interviews with 30 FloridaSDC participants (1). In that study, 10 interviewees reported gains in employment, five reported recent or planned transitions to independent housing, and nearly all reported pursuing valued roles in their communities. This study’s quantitative findings, from a sample that included nearly all eligible FloridaSDC participants, provides initial confirmatory evidence of findings reported for the smaller sample.
To the best of our knowledge, this study had the longest observation period of any mental health self-direction research. Self-direction is a complex intervention and requires a high level of participant engagement (1). Developing person-centered plans and working with a budget takes time. As such, gains may not be fully realized until after months or years of participation. This study’s three-to-five-year observation period may provide a more representative picture of program effects compared with earlier studies.
Conclusions
This study adds to a small but growing body of literature addressing mental health self-direction’s effects. Compared with nonparticipants, self-directing participants were more likely to improve, or maintain at high levels, engagement in paid work and independent housing. We note, however, that there were marked case mix differences between groups, and there was uncertainty about whether the analytic approach adequately accounted for these differences. Future research should seek to corroborate these findings using more robust methods and might involve additional outcome domains, such as service use and social connectedness, as well as alternative measures for employment, community integration, and independent living. Future work should involve mixed methods and implementation science approaches to explore issues associated with program reach (for example, participant recruitment and formal and informal selection criteria) to further understand the specific mechanisms within self-direction that support recovery. For example, participants’ relationships with their support brokers have been found to be critical for facilitating gains in self-esteem and independence, which were identified as core drivers in recovery (1). Studies establishing deeper and more nuanced understanding of program effectiveness could inform future research on program- and system-level factors. Future research should also examine cost implications of self-direction across health and social service systems; such information would support mental health leadership in expanding the use of self-direction to enhance self-sufficiency and quality of life.
References
1.
Croft B, Parish S: Participants’ assessment of the impact of behavioral health self-direction on recovery. Community Mental Health Journal 52:781–792, 2016
2.
Spaulding-Givens JC, Lacasse JR: Self-directed care: participants’ service utilization and outcomes. Psychiatric Rehabilitation Journal 38:74–80, 2015
3.
Cook JA, Shore SE, Burke-Miller JK, et al: Participatory action research to establish self-directed care for mental health recovery in Texas. Psychiatric Rehabilitation Journal 34:137–144, 2010
4.
Snethen G, Bilger A, Maula EC, et al: Exploring personal medicine as part of self-directed care: expanding perspectives on medical necessity. Psychiatric Services 67:883–889, 2016
5.
Croft B, Simon-Rusinowitz L, Loughlin D, et al: An Environmental Scan of Self-Direction in Behavioral Health: Summary of Major Findings. Chestnut Hill, MA, Boston College National Resource Center for Participant-Directed Services, 2013
6.
Croft B, Wang K, Cichocki B, et al: The emergence of mental health self-direction: an international learning exchange. Psychiatric Services 68:88–91, 2017
7.
Webber M, Treacy S, Carr S, et al: The effectiveness of personal budgets for people with mental health problems: a systematic review. Journal of Mental Health 23:146–155, 2014
8.
Adler NE, Stead WW: Patients in context: EHR capture of social and behavioral determinants of health. New England Journal of Medicine 372:698–701, 2015
9.
Achieving the Promise: Transforming Mental Health Care in America. Pub no SMA-03-3832. Rockville, Md, Department of Health and Human Services, President’s New Freedom Commission on Mental Health, 2003
10.
SAMHSA: Description of a Good and Modern Addictions and Mental Health Service System. Rockville, MD, Substance Abuse and Mental Health Services Administration, 2011
11.
Iacus S, King G, Porro G: Causal inference without balance checking: coarsened exact matching. Political Analysis 20:1–24, 2011
12.
Blackwell M, Iacus S, King G, et al: CEM: coarsened exact matching in Stata. Stata Journal 9:524–546, 2009
13.
Iacus S, King G, Porro G: cem: coarsened exact matching software. Journal of Statistical Software 30:1–27, 2008
14.
Shadish WR, Cook TD, Campbell DT: Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston, Houghton, Mifflin, 2002
15.
Cook JA, Russell C, Grey DD, et al: A self-directed care model for mental health recovery. Psychiatric Services 59:600–602, 2008
16.
Hall RL: Report on the Effectiveness of the Self-Directed Care Community Mental Health Treatment Program as Required by s. 394.9084 FS. Tallahassee, Florida Department of Children and Families Mental Health Program Office, 2007
17.
Insufficient Information Available to Fully Assess the Success of the Self-Directed Care Program. Tallahassee, Office of the Florida Legislature, Office of Program Policy Analysis and Government Accountability, 2010
18.
Wolff N: Using randomized controlled trials to evaluate socially complex services: problems, challenges and recommendations. Journal of Mental Health Policy and Economics 3:97–109, 2000
Information & Authors
Information
Published In
Cover: Red Umbrella, by Milton Avery, 1945. Oil on canvas. Gift of Annalee Newman, Princeton Art Museum. Photo credit: Bruce M. White, Princeton University Art Museum/Art Resource. © The Milton Avery Trust/Artists Rights Society, New York City.
History
Received: 6 February 2017
Revision received: 18 July 2017
Revision received: 6 October 2017
Revision received: 23 January 2018
Revision received: 13 March 2018
Accepted: 23 March 2018
Revision received: 21 April 2018
Published online: 15 May 2018
Published in print: July 01, 2018
Keywords
Authors
Competing Interests
The authors report no financial relationships with commercial interests.
Funding Information
Robert Wood Johnson Foundation10.13039/100000867: #71920
National Institute on Alcohol Abuse and Alcoholism10.13039/100000027: 5T32AA007567-17
Support for the preparation of this article was provided to Dr. Croft by grant 71920 from the Robert Wood Johnson Foundation and grant 5T32AA007567-17 (doctoral training fellowship) from the National Institute on Alcohol Abuse and Alcoholism.Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).