Mental Health Self-Directed Care Financing: Efficacy in Improving Outcomes and Controlling Costs for Adults With Serious Mental Illness
Abstract
Objective:
Self-directed care allows individuals with disabilities and elderly persons to control public funds to purchase goods and services that help them remain outside institutional settings. This study examined effects on outcomes, service costs, and user satisfaction among adults with serious mental illness.
Methods:
Public mental health system clients were randomly assigned to self-directed care (N=114) versus services as usual (N=102) and assessed at baseline and 12 and 24 months. The primary outcome was self-perceived recovery. Secondary outcomes included psychosocial status, psychiatric symptom severity, and behavioral rehabilitation indicators. Mixed-effects random-regression analysis tested for longitudinal changes in outcomes between the two conditions. Differences in service costs were analyzed with negative binomial and zero-inflated negative binomial regression models.
Results:
Compared with the control group, self-directed care participants had significantly greater improvement over time in recovery, self-esteem, coping mastery, autonomy support, somatic symptoms, employment, and education. No between-group differences were found in total per-person service costs in years 1 and 2 or both years combined. However, self-directed care participants were more likely than control group participants to have zero costs for six of 12 individual services and to have lower costs for four. The most frequent nontraditional purchases were for transportation (21%), communication (17%), medical care (15%), residential (14%), and health and wellness needs (11%). Client satisfaction with mental health services was significantly higher among intervention participants, compared with control participants, at both follow-ups.
Conclusions:
The budget-neutral self-directed care model achieved superior client outcomes and greater satisfaction with mental health care, compared with services as usual.
Highlights
•
Self-directed care participants had significantly greater improvement over 24 months, compared with a control group, in recovery, self-esteem, coping mastery, autonomy support, somatic symptoms, employment, and education.
•
Although total per-person service costs did not differ between study conditions, self-directed care participants were more likely than control group participants to have zero costs for six out of 12 individual services and to have lower costs for four individual services.
•
Client satisfaction with mental health services was significantly higher among self-directed care participants, compared with the control group, at 1- and 2-year follow-up.
Over the past two decades, the federal Centers for Medicare and Medicaid Services (CMS) has promoted use of a consumer-directed, “money follows the person,” health care financing approach for use by individuals with a broad range of disabilities. Called self-directed care, this model gives individuals direct control over public funds to purchase health care services, supports, and material goods necessary for them to reside in the community rather than in inpatient or nursing facilities. Although use of this model to promote the recovery of people with serious mental illness is less common, multiple states are now developing mental health self-directed care initiatives, and interest in this approach is growing. Recently, consumer, advocacy, and service provider communities have called for greater use of self-directed care in mental health, as have federal agencies, including CMS (1), the Substance Abuse and Mental Health Services Administration (2), and the U.S. Department of Health and Human Services’ Office of Disability, Aging and Long-Term Care Policy (3). The purpose of this study was to conduct a randomized controlled trial of a mental health self-directed care program, assessing its effects on participant outcomes, service satisfaction, and service costs.
Development of Self-Direction in Health Care Contexts
Self-directed care is a model of consumer self-management of resources, originally developed for the long-term care of people with physical and developmental disabilities and elderly persons. It was first tested in a national evaluation funded by CMS called the Cash and Counseling Demonstration (4). For this demonstration, CMS authorized states to allow Medicaid beneficiaries to have direct control over cash expenditures for goods and services, so that participants could hire personal care attendants, purchase items, and make home modifications to help them live independently. Controlled trials were conducted in Arkansas, New Jersey, and Florida, in which participants were randomly assigned to self-directed care versus services as usual and interviewed at baseline and 9-month follow-up (5). Individuals were not eligible for the demonstration if their primary disability was a psychiatric disorder. Compared with those who received usual services, intervention participants were more satisfied with their care, had fewer unmet needs, and had equal or better health outcomes (6, 7). Cost neutrality prevailed by the second year, with negligible differences between spending on self-directed versus agency-based care (8). Other studies have shown that self-directed care promotes positive outcomes for adults with physical disabilities (9), elderly persons (10), young adults (11), and other groups of Medicaid recipients (12). CMS has increasingly encouraged states to use Medicaid dollars to implement this model, in response to rising costs in the health care field as well as advocacy from the disability rights and independent living movements (13).
Development of Mental Health Self-Directed Care
Despite its rapid uptake by states and inclusion in recent health care legislation, use of this model for people with serious mental illness has been sporadic. In a 2007 review, only seven states had mental health self-directed care programs (14). Yet evaluations of these programs have yielded encouraging results. Two separate pre-post evaluations of the District 4 FloridaSDC program compared outcomes in the year prior to program entry with the year after and found significant increases in number of days spent in the community (versus jails or hospitals) as well as increased levels of functioning (15, 16). A study of the FloridaSDC Circuit 20 program found better employment and residential outcomes among participants, compared with a matched control group (17). Finally, secondary analysis of data from nonelderly Cash and Counseling participants who used mental health services in the year prior to baseline (a proxy for mental illness) found that compared with those in the control group, self-directed care participants were less likely to have unmet needs and to experience adverse medical events, and more likely to be satisfied with services (18).
At the same time, concerns have been raised about the use of this model with individuals in mental health recovery (19). Common objections include the purported lack of capacity to self-direct one’s care among people with serious mental illness, lack of community acceptance of individual budgets, provider agency rules limiting the ability to self-direct services, and potential disruption to the service system (20–22). Another set of critiques faults the model for its neoliberal approach, arguing that it replaces “enforced collectivism” of an earlier era with “enforced individualism,” which makes people responsible for their own care in the context of inadequate fiscal and societal supports (23, 24).
Despite the foregoing objections, preliminary research evidence suggests that self-directed care shows great potential for improving recovery outcomes while costing no more than traditional service delivery. The conceptual model for our proposed intervention was drawn from research on self-determination theory (25, 26), which emphasizes the importance of autonomous motivation—stemming from a sense of volition, self-initiation, and personal endorsement—as a cause of lasting health behavior change (27). Autonomous motivation, in turn, is enhanced when individuals are in settings where others understand their perspectives, acknowledge their feelings, offer choices, and provide relevant information, called autonomy-supportive environments (28). Empirical studies show that people in such environments have a greater likelihood of changing and maintaining desired health behaviors (29, 30). We posit that self-directed care’s emphasis on providing information, encouraging choice, and honoring self-knowledge from lived experience with mental illness offers an autonomy-supportive environment that leads to greater likelihood of recovery and other positive outcomes.
The first goal of this intent-to-treat analysis was to evaluate efficacy by testing the null hypothesis of no difference between self-directed care and services as usual in changes over time in measures of recovery, clinical outcomes, and behavioral rehabilitation indicators. The second goal was to evaluate budget neutrality by testing a null hypothesis of no difference in service utilization and costs between the two conditions. The third goal was to evaluate self-directed care participants’ satisfaction with mental health services and compare it to that of the control group.
Methods
The Texas SDC program was developed through a partnership between the University of Illinois at Chicago’s Center on Mental Health Services Research and Policy and the Texas Department of State Health Services’ (DSHS) Mental Health and Substance Abuse Division, in collaboration with a group of community stakeholders that included people in recovery, service providers, family members, behavioral health authority officials, managed care staff, and advocacy organizations. This participatory process has been described in detail elsewhere (31). The program operated from May 2009 through May 2013.
Inclusion criteria were being an active Texas DSHS client; presence of serious mental illness consistent with federal Public Law 102–32 (32) and defined by the Texas DSHS (33) as “individuals with major depressive disorder (GAF ≤ 50), bipolar disorder, or schizophrenia and related disorders . . . with moderate to severe levels of need . . . who require intensive rehabilitation to increase community tenure, establish support networks, increase community awareness, and develop coping strategies in order to function effectively;” eligible for a comprehensive package of clinical and rehabilitation services known as service package 3; age 18 years or older; and able to understand spoken English. Exclusion criteria were cognitive impairment (defined as a score of ≥2 on a brief screening assessment of cognitive impairment); being homeless at the time of recruitment; and a history of violent behavior resulting in arrest and conviction in the past 10 years as revealed by a criminal background check.
Study Procedures
Potential participants were recruited at community mental health agencies by local research staff who were hired and trained by University of Illinois at Chicago investigators. Recruitment commenced in March 2009 and ended in September 2010. The nature of the self-directed care model and tasks required of research participants were described in detail prior to obtaining written informed consent with human subjects procedures approved by the institutional review boards of the University of Illinois at Chicago and the Texas DSHS. Random assignment was performed by interviewers from the University of Illinois at Chicago Survey Research Laboratory at the end of the baseline interviews by using a randomly generated allocation sequence that was programmed into computer-assisted interviewing software. This method permitted complete allocation concealment so that interviewers and respondents had no way of knowing each participant’s study condition until after assignment occurred. Information regarding participant characteristics and outcomes was obtained during baseline, 12-month, and 24-month telephone interviews. Utilization and cost data for traditional behavioral health services were obtained from the state’s electronic data warehouse (34), and data for nontraditional services costs came from the self-directed care program’s electronic administrative files. Client satisfaction data were gathered at the 12- and 24-month telephone interviews.
Measures
All measures used in the study were psychometrically validated. The primary outcome was participants’ perceived level of recovery from mental illness as measured by the Recovery Assessment Scale (35, 36). This 41-item measure yields a total score, along with scores on subscales that measure goal and success orientation, personal confidence and hope, reliance on others, willingness to ask for help, and not feeling dominated by symptoms. The Cronbach’s α for baseline total score was excellent at .93, with good to acceptable scores for subscales, including personal confidence and hope (α=.80), goal and success orientation (α=.79), willingness to ask for help (α=.76), reliance on others (α=.73), and not feeling dominated by symptoms (α=.60).
Secondary outcomes included changes in psychosocial statuses. The first was self-esteem, assessed by a subscale of the Empowerment Scale (37), which measures feelings of self-worth and confidence in general abilities, with a baseline α of .97. A second was the participant’s sense of personal control over important life outcomes, assessed by the Coping Mastery Scale (38) with a baseline α of .66. The third outcome was the extent to which participants felt that they were being served in autonomy-supportive environments—as measured by the Perceived Autonomy Support Scale (39), with a baseline α of .92.
Reduction in psychiatric and somatic symptoms was measured with the Brief Symptom Inventory’s Global Severity Index, which is designed to quantify an individual’s severity of illness (40). The inventory’s somatic subscale assesses physical manifestations of emotional distress, such as pain, weakness, or shortness of breath (41). We also assessed the rehabilitation outcomes of paid employment (using the U.S. Department of Labor’s definition of any work at all for pay or profit during a reference week) and participation in formal education or training programs (using the U.S. Department of Education’s definition of formal education as instruction provided in a system of schools, colleges, universities, and other formal education institutions). Finally, the Client Satisfaction Questionnaire (42) was administered at the 12- and 24-month follow-up points to assess satisfaction with mental health services among study participants in both study conditions, with a 12-month α of .95.
Service utilization and cost data were obtained from the electronic administrative records maintained by the state of Texas’s Data Warehouse. All study participants were eligible for the same package of public mental health services because of their Level of Care assessment conducted according to the Adult Texas Recommended Assessment Guidelines (31). These services included medication management, diagnostic assessment, case management, skills training, psychosocial rehabilitation, peer services, psychotherapy, substance abuse services, residential services, crisis services, inpatient treatment, and psychiatric medications. [Traditional service category definitions are presented in an online supplement to this article.]
Mental Health Self-Directed Care Intervention
The intervention was implemented in a managed behavioral health carve-out that offered an integrated, single system of care overseen by the managed care company Value Options. It served the medically indigent (those without public or private health insurance) and most Medicaid recipients residing in Dallas, Collin, Ellis, Hunt, Kaufman, Navarro, and Rockwall counties. The existence of a managed care waiver meant that the self-directed care program could be offered to service recipients selectively as a pilot program with an evaluation component. Use of a braided funding system already was in place for the carve-out, combining Medicaid, community mental health block grant, private foundation, and state general revenue funds.
The intervention followed the basic features of mental health self-directed care (15). Participants began by developing person-centered plans for recovery as mandated by CMS. Next, they created individual budgets with line items for the purchase of services and goods corresponding to plan goals, which were reviewed and approved by program management. Participants received assistance from self-directed care staff acting as service brokers, who helped them develop plans and budgets and then aided them in selecting and securing needed services and supports. Further details regarding the intervention can be found in a mental health self-directed care implementation manual (43).
Payment of providers and management of financial resources were done by the mental health carve-out’s managed care organization acting as fiscal intermediary. The size of each participant’s annual budget was $4,000, which was the regional average per-patient cost of mental health services, excluding inpatient treatment, crisis services, and medications. Each of the latter three services remained available to all participants through regular channels. Participants could budget for nontraditional purchases, which were defined as using mental health service providers outside the public system; purchasing services not usually offered in public mental health, such as massage therapy or nutritional counseling; and making service substitutions, such as a buying a bus pass instead of using an agency van or purchasing material goods, such as a cell phone, laptop, or job interview clothing. In all cases, expenditures were required to be tied directly to recovery goals and to be part of preapproved individual budgets. On average, participants developed their budgets within 1 month of program entry and made their first purchase within 4 weeks following budget approval. Over 24 months of study participation, 98% (N=112) of the self-directed care group purchased any services and 80% (N=91) made nontraditional purchases.
Statistical Analysis
We examined differences in participants’ baseline characteristics associated with randomized study condition by using chi-square and t tests. We analyzed outcome data with random-effects linear and logistic regression models. This approach statistically accommodates features of our data that are common in longitudinal analysis, including serial correlation, individual heterogeneity at baseline, missing observations, and time-varying measures. In our models, these characteristics were handled by use of repeated covariance specification, random intercepts, inclusion of all available data, and a linear measure of time. Each model included random intercept, intervention study condition, time (baseline, 12 months, and 24 months), and intervention study condition × time interaction (44). No additional covariates were included in the models, given the statistical equivalence of the study groups on baseline characteristics (described below).
Average per-person service costs by service type, year, and over both years were examined descriptively. Cost data are counts of dollars and have a distribution skewed to the right, with lower costs being more frequently represented than higher costs and frequent zero costs being common for specific services, such as hospitalization. Differences in total costs associated with study condition were analyzed by using generalized linear models with negative binomial distribution (45). Individual services were analyzed by using zero-inflated negative binomial regressions, which model, first, the likelihood of having zero costs in each service category (logistic component), second, the relative amount of costs among the nonzeros (negative binomial component), and third, an overall estimated treatment effect calculated from the negative binomial component with an unconditional logistic component (intercept only) (46). Finally, we compared mental health service satisfaction by study condition using chi-square and t tests. Data were analyzed in SAS 9.4 and SPSS 24.
Study Retention
A total of 217 individuals consented to participate in the study and were randomly assigned to the intervention (N=115) or control (N=102) conditions [see CONSORT diagram in the online supplement ]. One participant assigned to the intervention condition requested removal from the study, resulting in an analysis sample of 216 individuals. Most study participants (81%, N=175 of 216) completed the final (24-month) follow-up interview, and this proportion did not differ significantly by study condition (intervention group, 83%; control group, 79%). There was 3% mortality among participants over 24 months (N=7 of 216), with causes including congestive heart failure, renal failure, acute myocardial infarction, liver failure, and accident.
Fidelity
A self-directed care fidelity assessment was developed with the format and methods tested in the National Implementing Evidence-Based Practices Project (47), and both process and structural fidelity were assessed in three programmatic areas: staffing, organization and management, and service delivery. During 2-day site visits, an external team of self-directed care experts assessed the presence of critical model components by using information from a review of randomly selected case records; analysis of written policies and procedures; interviews with program staff, participants, fiscal intermediary staff, and service providers; observations of interactions between staff and participants; and observation of individual staff supervision as well as weekly staff meetings. Items were rated on a Likert scale, and scores were summed to arrive at a total score and subscores. Fidelity assessment team members first made independent ratings and then discussed and resolved any discrepancies to arrive at final scores. Results were shared with program staff and managers at the end of each visit, and actions to address any fidelity drift were planned, including consultation and refresher training. A total of seven site visits were conducted at 6-month intervals throughout the time of the program’s operation, starting December 2009 and ending December 2012. Total fidelity scores increased significantly over time (p<0.05), from an initial score of 76, representing “fair fidelity,” to a final score of 116, representing “exemplary” fidelity. Two of the three subareas—service delivery and organization/management—also showed significant improvement over time (regression estimate=2.22, p<0.05; regression estimate=4.01, p<0.001, respectively), and the third area, staffing, showed nonsignificant improvement.
Results
Participant Background Characteristics and Equivalence of Study Conditions
The 216 participants in the two conditions did not differ significantly (p<0.05) on measured baseline characteristics (Table 1). Mean and median age was 42. Half were white (50%), 48% were black, and 3% reported membership in other racial groups. Fifteen percent reported Hispanic-Latino ethnicity. Over half were female (62%), and two-thirds had at least a high school diploma or equivalent (66%). Almost half were living in their own home or apartment (47%), 13% were married or living with a partner, and 69% were parents. Only 12% were employed at study baseline, and 50% had annual incomes below $10,000. Almost half had a co-occurring physical condition or impairment (48%). On the basis of administrative data, about one-third (34%) had a diagnosis of schizophrenia, two-fifths (41%) had a diagnosis of major depressive disorder, and a quarter (26%) had a diagnosis of bipolar disorder. Around one-third were Medicaid beneficiaries (35%), and the others had no health insurance coverage, qualifying them for services as medically indigent. On average, participants incurred $4,204 in total mental health service costs in the year prior to study baseline, with no significant difference in mean cost by study condition.
| Total (N=216) | Self-directed care (N=114) | Control group (N=102) | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Characteristic | N | %a | N | %a | N | %a | Test statistic | df | p |
| Race | χ2=.93 | 2 | .630 | ||||||
| White-Caucasian | 107 | 50 | 60 | 53 | 47 | 46 | |||
| Black–African American | 103 | 48 | 51 | 45 | 52 | 51 | |||
| Other | 6 | 3 | 3 | 3 | 3 | 3 | |||
| Hispanic-Latino | 33 | 15 | 22 | 19 | 11 | 11 | χ2=3.02 | 1 | .091 |
| Female | 134 | 62 | 76 | 67 | 58 | 57 | χ2=2.20 | 1 | .138 |
| High school diploma or GED | 142 | 66 | 76 | 67 | 66 | 65 | χ2=.09 | 1 | .762 |
| Married or living with partner | 28 | 13 | 16 | 14 | 12 | 12 | χ2=.25 | 1 | .620 |
| Living in own home or apartment | 101 | 47 | 53 | 47 | 48 | 47 | χ2=.01 | 1 | .933 |
| Parent | 148 | 69 | 75 | 66 | 73 | 72 | χ2=.83 | 1 | .361 |
| Employed | 26 | 12 | 13 | 11 | 13 | 13 | χ2=.09 | 1 | .762 |
| Annual income of <$10,000 | 102 | 50 | 51 | 47 | 51 | 54 | χ2=1.13 | 1 | .289 |
| Physical condition or impairment | 104 | 48 | 57 | 50 | 47 | 46 | χ2=.33 | 1 | .565 |
| Diagnosis category | χ2=1.71 | 2 | .426 | ||||||
| Schizophrenia spectrum disorder | 73 | 34 | 38 | 33 | 35 | 34 | |||
| Major depressive disorder | 88 | 41 | 43 | 38 | 45 | 44 | |||
| Bipolar I or II disorder | 55 | 26 | 33 | 29 | 22 | 22 | |||
| Co-occurring substance abuse or dependence | 37 | 17 | 20 | 18 | 17 | 17 | χ2=.29 | 1 | .864 |
| Medicaid eligible in first 90 days of study participation | 74 | 35 | 42 | 37 | 32 | 32 | χ2=.48 | 1 | .564 |
| Age (M±SD) | 41.6±9.7 | 41.6±10.0 | 41.6±9.5 | t=–.02 | 1, 214 | .984 | |||
| Mental health services costs in year prior to study (M±SD $) | 4,204±3,553 | 4,390±3,288 | 3,996±3,833 | t=.81 | 1, 214 | .417 | |||
a
Percentages may sum to >100% because of rounding.
Multivariable Analysis of Outcomes
Table 2 presents the results of the random-effects linear and logistic regression analyses of primary and secondary outcomes. Statistically significant intervention × time interaction effects indicate that the intervention group had significantly better outcomes over time, compared with the control group, in the primary outcome of recovery. Compared with those in the control group, those assigned to self-directed care improved significantly over time in Recovery Assessment Scale total scores (estimate=4.27, p=0.009) and on two of its subscales: goal orientation (.71, p=0.007) and personal confidence (1.10, p=0.027). On the three remaining subscales, the positive differences in the intervention group compared with the control group fell short of statistical significance: reliance on others, willingness to ask for help, and no symptom domination.
| Outcome | Estimate | SE | p |
|---|---|---|---|
| Recovery Assessment Scale | |||
| Total score | |||
| Intercept | 153.40 | 2.05 | <.001 |
| Intervention | –5.53 | 2.82 | .051 |
| Time | .23 | 1.19 | .846 |
| Intervention × time | 4.27 | 1.61 | .009 |
| Goal orientation | |||
| Intercept | 20.67 | .33 | <.001 |
| Intervention | –.54 | .45 | .226 |
| Time | –.25 | .19 | .193 |
| Intervention × time | .71 | .26 | .007 |
| Personal confidence | |||
| Intercept | 33.79 | .58 | <.001 |
| Intervention | –1.90 | .81 | .019 |
| Time | .13 | .33 | .705 |
| Intervention × time | 1.10 | .46 | .027 |
| Reliance on others | |||
| Intercept | 14.90 | .32 | <.001 |
| Intervention | –.32 | .44 | .462 |
| Time | –.02 | .17 | .898 |
| Intervention × time | .46 | .24 | .052 |
| Willingness to request help | |||
| Intercept | 12.50 | .22 | <.001 |
| Intervention | –.47 | .30 | .124 |
| Time | –.09 | .15 | .553 |
| Intervention × time | .35 | .21 | .089 |
| Not feeling dominated by symptoms | |||
| Intercept | 9.22 | .25 | <.001 |
| Intervention | –.49 | .35 | .161 |
| Time | .35 | .17 | .044 |
| Intervention × time | .42 | .24 | .075 |
| Self-esteem | |||
| Intercept | 24.84 | .53 | <.001 |
| Intervention | –.79 | .73 | .279 |
| Time | .49 | .31 | .109 |
| Intervention × time | .90 | .42 | .031 |
| Coping mastery | |||
| Intercept | 2.68 | .05 | <.001 |
| Intervention | –.08 | .07 | .237 |
| Time | –.02 | .03 | .501 |
| Intervention × time | .12 | .04 | .007 |
| Perceived autonomy support | |||
| Intercept | 4.68 | .16 | <.001 |
| Intervention | .20 | .21 | .349 |
| Time | .07 | .10 | .456 |
| Intervention × time | .29 | .13 | .030 |
| Brief Symptom Inventory | |||
| General severity index | |||
| Intercept | 73.41 | .88 | <.001 |
| Intervention | .24 | 1.21 | .844 |
| Time | –2.14 | .49 | <.001 |
| Intervention × time | –.06 | .67 | .931 |
| Somatic symptoms | |||
| Intercept | 65.77 | 1.04 | <.001 |
| Intervention | 1.81 | 1.43 | .206 |
| Time | .35 | .53 | .503 |
| Intervention × time | –2.16 | .72 | .003 |
| Outcome | OR | 95% CI | p |
| Employed (reference: not employed) | |||
| Intercept | –3.05 | .51 | <.001 |
| Intervention | 1.27 | .43–3.75 | .670 |
| Time | .62 | .34–1.15 | .131 |
| Intervention × time | 2.19 | 1.01–4.74 | .046 |
| Enrolled in school or class (reference: not enrolled) | |||
| Intercept | –2.94 | .55 | <.001 |
| Intervention | .38 | .11–1.27 | .115 |
| Time | .39 | .17–.87 | .021 |
| Intervention × time | 4.14 | 1.50–11.44 | .006 |
a
Random-effects linear and logistic regression models. Time, baseline, 12 months, and 24 months.
Turning to measures of psychosocial health, compared with the control group, the intervention group improved significantly over time in level of self-esteem (.90, p=0.031). Self-directed care participants also showed significantly greater improvement over time than those in the control group in coping mastery (.12, p=0.007). Finally, compared with the control condition, intervention participants improved significantly over time in the extent to which they perceived their service delivery environment as supportive of their personal autonomy (.29, p=0.030).
The two study groups did not differ significantly in global psychiatric symptom severity; however, the intervention participants had significantly lower somatic symptom severity over time than the control participants (−2.16, p=0.003). Regarding rehabilitation outcomes, intervention participants were more than twice as likely as those in the control group to be employed over time (odds ratio [OR]=2.19, p=0.046). More specifically, the proportion of employed intervention participants rose from 11% at baseline to 18% at 12 months and 20% at 24 months, while the corresponding proportions for control group participants were 6%, 5%, and 12%. In addition, self-directed care participants were over four times as likely as control group participants to be enrolled in formal education (OR=4.14, p=0.006). The proportion of intervention participants taking classes was 6% at baseline, 5% at 12 months, and 12% at 24 months, and the corresponding control group proportions were 10%, 6%, and 1%.
Service Use and Costs
The intervention group had lower total mean costs per person than control participants in the first year ($2,998 versus $3,189) and second year ($2,241 versus $2,303) of study participation, and in both years combined ($5,240 versus $5,493). [A table in the online supplement presents costs by service type, study condition, and year of study participation.]
Regarding individual service categories, in the first and second years and in both years combined, comparison of means shows that the intervention group spent less on average per person, compared with the control group, on skills training, psychosocial rehabilitation, case management, inpatient hospitalization, psychiatric crisis services, substance abuse treatment, medication management, and medications. Over the first and second years and both years combined, compared with the control group, the intervention group spent more on average on psychotherapy and peer services. Regarding diagnostic services, in the first year and in both years combined, the intervention group spent more on average, compared with the control group, and spending was the same in the second year. Regarding residential services, the intervention group spent more than the control group in the first year, less in the second year, and more over both years combined. Only the intervention group could make nontraditional expenditures, which averaged $679 per person in the first year, $667 per person in the second year, and $1,346 per person over both years combined.
Table 3 shows the proportion of participants using services and the results of negative binomial and zero-inflated negative binomial analyses testing for study condition effects on total and individual service costs. Four pieces of information are presented for each service category in years 1 and 2 and in both years combined. The first is the proportion of participants that used a service. The second is the likelihood of being a nonuser of the service and thus of incurring zero costs for that service, among all study participants. The third is the differential magnitude of costs for a service, among only users of that service. The fourth is the self-directed care treatment effect, combining both likelihood of incurring zero cost and magnitude of cost.
| Variable | Total services | Psychotherapy | Skills training | Psychosocial rehabilitation | Case management | Inpatient | Substance abuse | Medication management | Medication |
|---|---|---|---|---|---|---|---|---|---|
| Year 1 | |||||||||
| Participants who used services (%) | 97 | 22 | 11 | 49 | 11 | 6 | 7 | 82 | 88 |
| Likelihood of incurring zero costs (SDC versus control group) | |||||||||
| Estimate | –1.486 | 1.350 | .755 | .912 | .777 | .553 | .796 | 1.486 | |
| p | <.001 | .006 | .007 | .046 | .140 | .312 | .036 | .045 | |
| Difference in costs among service users only (SDC versus control group) | |||||||||
| Estimate | 1.092 | –.796 | –.297 | .638 | –.014 | –1.017 | –.375 | .138 | |
| p | <.001 | .011 | .124 | .216 | .965 | .089 | <.001 | .506 | |
| Treatment effect | |||||||||
| Estimate | –.062 | 1.094 | –.797 | –.295 | .624 | .013 | –1.019 | –.475 | –.163 |
| p | .718 | <.001 | .011 | .156 | .224 | .970 | .090 | <.001 | .436 |
| Year 2 | |||||||||
| Participants who used services (%) | 86 | 16 | 9 | 26 | 13 | 4 | 5 | 58 | 71 |
| Likelihood of incurring zero costs (SDC versus control group) | |||||||||
| Estimate | –1.900 | 1.930 | .383 | 1.635 | .637 | 1.141 | .326 | .023 | |
| p | <.001 | .003 | .223 | <.001 | .392 | .010 | .243 | .946 | |
| Difference in costs among service users only (SDC versus control group) | |||||||||
| Estimate | .750 | .204 | –.014 | –1.012 | –.692 | .605 | –.443 | –.286 | |
| p | .136 | .714 | .956 | .040 | .045 | .457 | .001 | .271 | |
| Treatment effect | |||||||||
| Estimate | –.018 | .765 | .201 | –.015 | –1.07 | –.692 | .599 | –.443 | –.286 |
| p | .933 | .127 | .718 | .955 | .033 | .045 | .461 | .001 | .269 |
| Year 1 and year 2 combined | |||||||||
| Participants who used services (%) | 91 | 20 | 10 | 37 | 12 | 5 | 6 | 70 | 80 |
| Likelihood of incurring zero costs (SDC versus control group) | |||||||||
| Estimate | –1.602 | 1.484 | .755 | 1.183 | .774 | .532 | .731 | 1.759 | |
| p | <.001 | <.001 | .007 | .001 | .140 | .245 | .076 | .119 | |
| Difference in costs among service users only (SDC versus control group) | |||||||||
| Estimate | 1.134 | –.490 | –.297 | .076 | –.014 | –.606 | –.439 | –.189 | |
| p | <.001 | .151 | .124 | .855 | .965 | .233 | <.001 | .371 | |
| Treatment effect | |||||||||
| Estimate | –.047 | 1.145 | –.494 | –.298 | –.014 | –.014 | –.608 | –.439 | –.212 |
| p | .789 | <.001 | .148 | .122 | .965 | .965 | .231 | <.001 | .316 |
a
The study conditions were self-directed care (SDC) and a control group. Total costs were analyzed with negative binomial models, and costs for individual services were analyzed with zero-inflated negative binomial models. Four services for which the differences between study conditions were not statistically significant are not shown: diagnostic, peer, crisis, and residential services.
Almost all participants (97%) used services in the first year of study participation, decreasing to 86% in year two, and averaging 91% over both years combined. As shown in the first column of Table 3, for total costs, there was an overall negative but nonsignificant treatment effect of intervention condition compared with control condition in total costs in year 1 (–.06), year 2 (–.02) and both years combined (–.05).
Turning to use of individual services, self-directed care participants were more likely than control participants to incur zero costs for both skills training and case management in years 1 and 2 and both years combined. Intervention participants were more likely than control participants to incur zero costs for psychosocial rehabilitation in year 1 and in both years combined. Intervention participants were more likely than control participants to incur zero costs for substance abuse treatment in year 2 and for medications and medication management in year 1. Conversely, intervention participants were more likely than control participants to incur costs for psychotherapy in years 1 and 2 and both years combined.
Turning next to magnitude of costs, among users of medication management, self-directed care participants had significantly lower costs, compared with control participants, in years 1 and 2 and both years combined. Among users of skills training, intervention participants had significantly lower costs, compared with control participants, in year 1. Among users of case management and users of inpatient services, intervention participants had significantly lower costs, compared with control participants, in year 2.
Turning finally to treatment effect, which summarizes the combined impact of self-directed care on incurring zero costs and cost magnitude, self-directed care participation significantly lowered costs for skills training in year 1, case management in year 2, inpatient services in year 2, and medication management in years 1 and 2 and both years combined. Conversely, self-directed care participation significantly increased costs for psychotherapy in year 1 and in both years combined.
Nontraditional Services and Material Goods
Figure 1 shows the types of nontraditional services that self-directed care participants purchased through their person-centered budgets. The 114 participants spent a total of $153,498 over 2 years. The expenditures were for transportation (21%); communication (17%); medical care (15%); residential (14%); health and wellness (11%); documents and administrative fees (7%); clothes and grooming (6%); vocational or educational pursuits (4%); leisure, social activities, and recreation (4%); and food-groceries (1%) [see the online supplement for definitions of nontraditional services and expenditure categories].
FIGURE 1. Expenditures on nontraditional services by 114 participants in a mental health self-directed care programa
aPercentages of total expenditures on nontraditional services by the 114 participants over 2 years. Total expenditures, $153,498.
Client Satisfaction
Comparison of scores on the Client Satisfaction Questionnaire showed that compared with control group participants, self-directed care participants had significantly higher satisfaction with their mental health services. Total satisfaction scores were significantly higher at both 12 months (t=−2.97, df=175, p=0.003) and 24 months after study baseline (t=−3.75, df=173, p<0.001). At 24 months, 60% (N=56) of intervention participants rated themselves as very satisfied with the mental health services currently received, compared with 38% (N=31) of control group participants, whereas only 10% (N=9) of self-directed care participants reported being very or somewhat dissatisfied with their mental health services, compared with 26% (N=21) of control group participants (χ2=11.08, df=3, p=0.011).
Discussion
In this randomized controlled trial, compared with individuals who received services as usual, self-directed care participants experienced significantly greater improvement in the primary outcome of self-perceived recovery. The difference was particularly notable regarding the recovery dimension of pursuing and attaining life goals and the dimension of having confidence in one’s ability to manage one’s illness while building a meaningful life. Self-directed care participants also had significantly better outcomes, compared with control group participants, on the secondary outcomes of self-esteem, coping mastery, and autonomy support. These three findings confirm the importance of service self-direction in enhancing the psychological health that often underpins successful performance of adult roles and the ability to deal with challenges stemming from mental illness. Perhaps related to self-management of emotional distress, self-directed care participants reported significantly greater improvement than controls in physical manifestations of psychological distress, such as dizziness, pain, nausea, shortness of breath, and numbness.
Compared with individuals in the control group, self-directed care participants were significantly better able to attain two behavioral rehabilitation outcomes: employment over time, which may be related to their greater self-confidence, self-esteem, and feelings of mastery; and enrollment in formal classes, which may be related to their heightened ability to pursue goals regarding future achievement and learning. Of note, the employment finding echoes superior employment outcomes found in the evaluation of another mental health self-directed care program (17).
Intervention participants reported higher satisfaction with the services they were receiving, compared with their counterparts in the control condition. This is a consistent finding of self-directed care programs across different types of programs, including mental health programs (17, 48) and programs for people with other types of disability (49, 50). The emphasis in self-directed care on increasing service choices and on instilling a sense of personal responsibility and pride for being able to manage one’s own recovery is thought to contribute to higher satisfaction with assistance received (15, 50).
Importantly, these superior outcomes were achieved in the context of service budget neutrality. Our intent-to-treat analysis found that self-directed care participants had lower total service costs on average, compared with control group participants, and that these differences were not significant, indicating that services used by participants in self-directed care cost no more than services they received via the traditional system of directly funding agencies. In terms of individual service costs, the self-directed care group was significantly more likely than the control group to have zero cost for six of the service types examined. The services for which zero costs were more likely to be incurred included case management, skills training, psychosocial rehabilitation, substance abuse treatment, medication management, and treatment with medications.
In addition to being more likely to incur zero costs for some services, the self-directed care group had lower costs, compared with the control group, for four of the individual service types studied. One of these was the intensive and costly service of inpatient treatment. Although no differences were found by study condition in the occurrence of psychiatric hospitalization, among those who used this service, the self-directed care group incurred significantly lower costs, compared with the control group, in the second study year. Inpatient services were used by only 5% of all participants over the entire study period; however, lower costs were also found for the frequently used service of medication management, for which costs were incurred by 70% of all respondents. Here, self-directed care participants incurred significantly lower medication management costs, compared with control participants, in study years 1 and 2 and in both years combined. In addition, self-directed care participants incurred significantly lower costs than control participants for case management in study year 2 and for skills training in study year 1.
There was, however, one service that the self-directed care group was significantly more likely than the control group to use and for which they incurred significantly higher costs. This was psychotherapy, used by about 20% of all study participants over the study period. It is interesting to speculate why this was the only service chosen more often by self-directed care participants and why they spent significantly more for it, compared with control group users of psychotherapy. If, as claimed by some, self-direction reveals consumer choice more accurately in a “free market” context (15), perhaps this finding highlights an important preference. If so, public systems might consider including psychotherapy in the available service array. It may be that as self-directed care participants’ self-esteem, self-confidence, and coping mastery improved, they were more willing and able to engage in psychotherapy. Another explanation is that these kinds of improvement may have stemmed from the self-directed care groups’ greater use of psychotherapy. Finally, it is possible that individuals used psychotherapy in different ways and that it had various effects during different phases of their recovery.
Descriptive analysis of nontraditional purchases revealed that the most common were for needs related to transportation, communication, medical expenses, housing, and health and wellness. These findings were surprisingly consistent with those from the study of a mental health self-directed care program in Pennsylvania, in which the most common nontraditional expenditures were related to transportation, maintaining health, diet and fitness, and handling stress (51), and with those from a study of a mental health self-directed care program in Florida, in which the most common nontraditional expenditures were in the categories of fitness, wellness, medical needs, transportation, and self-help (15). As others have noted, the consistency of these nontraditional purchases suggests that services and supports available from the traditional mental health system are inadequate to meet consumer needs (16). These results also suggest that participants have insight into what would best support their recovery and that enhancing their range of choices facilitates the development of the personal medicine necessary to enhance their own mental health (51).
The study had some limitations. Although randomization resulted in study groups that were equivalent in demographic and other baseline characteristics and service costs, we may not have measured other factors that influenced participant outcomes and costs. In addition, although the follow-up rate was good in both conditions, we did not have a measure of participant engagement in services, such as number of missed appointments or quality of provider-recipient relationships, that might have influenced outcomes. Also, we were not able to compare administrative costs of the two conditions, because the state did not have a waiver authority with special terms and conditions permitting Medicaid funding for brokerage and fiscal intermediary services. Another limitation was our inability to capture the occurrence or costs of any services that were delivered outside the DSHS system. Finally, many of the study’s outcomes were measured by self-report and were thus subject to potential biases and distortions due to factors such as poor memory or social desirability.
Conclusions
Our overall conclusion is that the self-directed care model achieved superior client outcomes for no greater service delivery expenditures than those resulting from the traditional service delivery system. Giving clients control over their service delivery dollars did not cause them to avoid traditional services such as psychiatric medication or psychotherapy and did not lead to fraud or misuse of funds. With support from program staff, individuals with serious mental illnesses were able to develop person-centered recovery plans, formulate budgets with line items corresponding to goals in these plans, and spend money responsibly and effectively. Such findings point to the need for further study of this model on a larger scale, in different mental health populations, across varying geographic locations, and in systems where Medicaid allows for full implementation of self-direction as a home- and community-based service. Research also is needed on the specific pathways by which differential use of goods and services in this model leads to positive rehabilitation outcomes, such as employment and educational advancement, as well as psychosocial outcomes, including enhanced self-esteem and coping mastery. Only then can we gauge the true potential of this model for furthering recovery from psychiatric disability.
Acknowledgments
The authors thank the following individuals and organizations for their contributions to the study: Andrew P. Batteiger, B.A., Holly Brock, M.Ed., L.P.C., Larisa A. Burke, M.P.H., Tina M. Carter, B.A., Jamie Cook, B.A., Cheryl Gayles, M.A., Wesley Gray, M.A., Melinda Hicks, Ph.D., Rachel T. MacFarlane, Ph.D., Brittony McNaughton, M.A., L.P.C., Luis Moreno, B.S.W., Mark Ohrtman, M.A., M.Ed., Dionne Peniston, Ph.D., Anna Salazar, A.A., Katherine Scheib, M.A., Richard K. Scotch, Ph.D., Dong Tran, M.S., Tommy Warnick, M.A., UIC Survey Research Laboratory, Mental Health America of Greater Dallas, and the North Texas Behavioral Health Authority.
Footnote
The contents of this article do not represent the policy of any federal agency or philanthropic organization and the reader should not assume endorsement by the Federal Government or private funders.
Supplementary Material
File (appi.ps.201800337.ds001.docx)
- Download
- 52.10 KB
File (appi.ps.201800337.ds001.pdf)
- View/Download
- 63.58 KB
References
1.
Medicaid Program; Community First Choice Option: Final Rule. Fed Regist 2012; 77:26828–26903
2.
Shen C, Mahoney KJ, Smyer MA, et al: Self-Directed Care in Mental Health: Learnings From the Cash & Counseling Demonstration Evaluation. Pub no SMA-10-4522. Rockville, MD, Department of Health and Human Services, 2010
3.
Slade E: Feasibility of Expanding Self-Directed Services to People With Serious Mental Illness. Washington, DC, US Department of Health and Human Services, Office of Disability, Aging, and Long-Term Care Policy, 2007. https://aspe.hhs.gov/system/files/pdf/108481/ExpSDSFeas.pdf
4.
Benjamin AE, Fennell ML: Putting the consumer first: an introduction and overview. Health Serv Res 2007; 42:353–361
5.
Foster L, Brown R, Phillips B, et al: Improving the quality of Medicaid personal assistance through consumer direction. Health Aff 2003; W3(Suppl Web Exclusives):162–175
6.
Carlson BL, Foster L, Dale SB, et al: Effects of Cash and Counseling on personal care and well-being. Health Serv Res 2007; 42:467–487
7.
Dale SB, Brown RS: How does Cash and Counseling affect costs? Health Serv Res 2007; 42:488–509
8.
Dale S, Brown R, Phillips B, et al: The effects of Cash and Counseling on personal care services and Medicaid costs in Arkansas. Health Aff 2003; W3(Suppl Web Exclusives):566–575
9.
Benjamin AE, Matthias R, Franke TM: Comparing consumer-directed and agency models for providing supportive services at home. Health Serv Res 2000; 35:351–366
10.
Benjamin AE, Matthias RE: Age, consumer direction, and outcomes of supportive services at home. Gerontologist 2001; 41:632–642
11.
Beatty PW, Richmond GW, Tepper S, et al: Personal assistance for people with physical disabilities: consumer-direction and satisfaction with services. Arch Phys Med Rehabil 1998; 79:674–677
12.
Doty P, Kasper J, Litvak S: Consumer-directed models of personal care: lessons from Medicaid. Milbank Q 1996; 74:377–409
13.
Croft B, Wang K, Cichocki B, et al: The emergence of mental health self-direction: an international learning exchange. Psychiatr Serv 2017; 68:88–91
14.
Alakeson V: The Contribution of Self-Direction to Improving the Quality of Mental Health Services. Washington, DC, US Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation, 2007
15.
Cook JA, Russell C, Grey DD, et al: A self-directed care model for mental health recovery. Psychiatr Serv 2008; 59:600–602
16.
Spaulding-Givens JC, Lacasse JR: Self-directed care: participants’ service utilization and outcomes. Psychiatr Rehabil J 2015; 38:74–80
17.
Croft B, İsvan N, Parish SL, et al: Housing and employment outcomes for mental health self-direction participants. Psychiatr Serv 2018; 69:819–825
18.
Shen C, Smyer MA, Mahoney KJ, et al: Does mental illness affect consumer direction of community-based care? Lessons from the Arkansas Cash and Counseling program. Gerontologist 2008; 48:93–104
19.
Stefan S: Competence Issues in Self-Directed Care. Newton, MA, Center for Public Representation, 2004
20.
Koyanagi C, Alfano E, Carty L: In the Driver’s Seat: A Guide to Self-Directed Mental Health Care. Washington, DC, Bazelon Center for Mental Health Law and the UPENN Collaborative on Community Integration, 2008
21.
Alakeson V: Self-directed care for adults with serious mental illness: the barriers to progress. Psychiatr Serv 2008; 59:792–794
22.
Cook JA, Morrow M, Battersby L: Intersectional policy analysis of self-directed mental health care in Canada. Psychiatr Rehabil J 2017; 40:244–251
23.
Kendall S, Cameron A: Personalisation of adult social care: self‐directed support and the choice and control agenda. Br J Learn Disabil 2014; 42:264–271
24.
Roulstone A, Morgan H: Neo-liberal individualism or self-directed support: are we all speaking the same language on modernising adult social care? Soc Policy Soc 2009; 8:333–345
25.
Ryan RM, Deci EL: Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol 2000; 55:68–78
26.
Ryan RM, Patrick H, Deci EL, et al: Facilitating health behaviour change and its maintenance: interventions based on self-determination theory. Eur Health Psychol 2008; 10:2–5
27.
Reeve J, Nix G, Hamm D: Testing models of the experience of self- determination in intrinsic motivation and the conundrum of choice. J Educ Psychol 2003; 95:375–392
28.
Williams GC, Rodin GC, Ryan RM, et al: Autonomous regulation and long-term medication adherence in adult outpatients. Health Psychol 1998; 17:269–276
29.
Roemmich JN, Lambiase Ms MJ, McCarthy TF, et al: Autonomy supportive environments and mastery as basic factors to motivate physical activity in children: a controlled laboratory study. Int J Behav Nutr Phys Act 2012; 9:16
30.
Moustaka FC, Vlachopoulos SP, Kabitsis C, et al: Effects of an autonomy-supportive exercise instructing style on exercise motivation, psychological well-being, and exercise attendance in middle-age women. J Phys Act Health 2012; 9:138–150
31.
Cook JA, Shore SE, Burke-Miller JK, et al: Participatory action research to establish self-directed care for mental health recovery in Texas. Psychiatr Rehabil J 2010; 34:137–144
32.
Center for Mental Health Services, Substance Abuse and Mental Health Services Administration: Final notice. Federal Register Notice 1993; 58:29422–29425
33.
Local Planning and Network Development Service Package Definitions. Austin, Texas Department of State Health Services, 2010. http://www.dshs.texas.gov/mhcommunity/LPND/definitions.shtm#SP1-A
34.
Cook JA, Ruggiero K, Shore S, et al: Public-academic collaboration in the application of evidence-based practice in Texas mental health system redesign. Int J Ment Health 2007; 36:36–49
35.
Giffort D, Schmook A, Woody C, et al: Recovery Assessment Scale. Cambridge, MA, Human Services Research Institute, 1995
36.
Corrigan PW, Salzer M, Ralph RO, et al: Examining the factor structure of the Recovery Assessment Scale. Schizophr Bull 2004; 30:1035–1041
37.
Rogers ES, Chamberlin J, Ellison ML, et al: A consumer-constructed scale to measure empowerment among users of mental health services. Psychiatr Serv 1997; 48:1042–1047
38.
Pearlin LI, Schooler C: The structure of coping. J Health Soc Behav 1978; 19:2–21
39.
Williams GC, Wiener MW, Markakis KM, et al: Medical students’ motivation for internal medicine. J Gen Intern Med 1994; 9:327–333
40.
Derogatis LR: Brief Symptom Inventory: Administration, Scoring, and Procedures Manual. New York, Pearson Education Publishing, 1993
41.
Piersma HL, Boes JL, Reaume WM: Unidimensionality of the Brief Symptom Inventory (BSI) in adult and adolescent inpatients. J Pers Assess 1994; 63:338–344
42.
Larsen DL, Attkisson CC, Hargreaves WA, et al: Assessment of client/patient satisfaction: development of a general scale. Eval Program Plann 1979; 2:197–207
43.
Cook JA, Curtis L, Jonikas JA, et al: Self-Directed Care Implementation Manual: A Comprehensive Mental Health Program Guide. Chicago, University of Illinois at Chicago, Center on Mental Health Services Research and Policy, 2015. https://www.center4healthandsdc.org/sdc-manual.html
44.
Gibbons RD, Hedeker D, DuToit S: Advances in analysis of longitudinal data. Annu Rev Clin Psychol 2010; 6:79–107
45.
Gardner W, Mulvey EP, Shaw EC: Regression analyses of counts and rates: Poisson, overdispersed Poisson, and negative binomial models. Psychol Bull 1995; 118:392–404
46.
Elhai JD, Calhoun PS, Ford JD: Statistical procedures for analyzing mental health services data. Psychiatry Res 2008; 160:129–136
47.
McHugo GJ, Drake RE, Whitley R, et al: Fidelity outcomes in the National Implementing Evidence-Based Practices Project. Psychiatr Serv 2007; 58:1279–1284
48.
Croft B, Parish S: Participants’ assessment of the impact of behavioral health self-direction on recovery. Community Ment Health J 2016; 52:781–792
49.
Young HM, Sikma SK: Self-directed care: an evaluation. Policy Polit Nurs Pract 2003; 4:185–195
50.
Schore J, Foster L, Phillips B: Consumer enrollment and experiences in the Cash and Counseling program. Health Serv Res 2007; 42:446–466
51.
Snethen G, Bilger A, Maula EC, et al: Exploring personal medicine as part of self-directed care: expanding perspectives on medical necessity. Psychiatr Serv 2016; 67:883–889
Information & Authors
Information
Published In
Cover: XXXX
History
Received: 16 July 2018
Revision received: 21 September 2018
Accepted: 14 November 2018
Published online: 11 January 2019
Published in print: March 01, 2019
Keywords
Authors
Competing Interests
The authors report no financial relationships with commercial interests.
Funding Information
National Institute on Disability and Rehabilitation Research10.13039/100006663: H133B050003
Meadows Foundation10.13039/100000914: 2011110176
National Institute on Disability, Independent Living, and Rehabilitation Research10.13039/100009157: 90RT5038
Center for Mental Health Services10.13039/100005268: 5 U79 SM057485-05
This research was funded by a cooperative agreement between the National Institute on Disability and Rehabilitation Research, U.S. Department of Education, and the Center for Mental Health Services, Substance Abuse and Mental Health Services Administration (SAMHSA) (H133B050003B); a grant from the Center for Mental Health Services, SAMHSA (U79 SM057485-05); a grant from the Meadows Foundation (2010110162); and a cooperative agreement between the National Institute on Disability, Independent Living, and Rehabilitation Research, Administration for Community Living, U.S. Department of Health and Human Services, and the Center for Mental Health Services, SAMHSA (90RT5038).Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).