Outcomes From Consumer-Operated and Community Mental Health Services: A Randomized Controlled Trial
Abstract
Objective:
Hierarchically organized board-and-staff-run consumer-operated service programs (COSPs) are viewed as organizations that promote recovery while working in concert with community mental health agencies (CMHAs). This study's objective was to determine the effectiveness of such combined services for people with serious mental illness.
Methods:
A board-and-staff-run consumer-operated drop-in center and colocated CMHA provided the context for the randomized clinical trial. In a weighted sample, 139 new clients seeking help from the CMHA were randomly assigned to agency-only service or to a combination of COSP and CMHA services. Client-members were assessed at baseline and eight months on a measure of symptom severity and on four recovery-focused outcome measures: personal empowerment, self-efficacy, independent social integration, and hopelessness. All scales used have high reliability and well-established validity. Differences in outcome by service condition were evaluated with multivariate analysis of covariance via dummy variable regression. Change scores on the five outcomes were the dependent variables. The covariates for the multivariate analysis included baseline status on each outcome measure and service condition between-group demographic differences.
Results:
Results indicated that significant changes in three recovery-focused outcomes were associated with service condition across time: social integration (p<.001), personal empowerment (p<.006), and self-efficacy (p<.001). All changes favored the CMHA-only condition. Neither symptomology nor hopelessness differed by service condition across time.
Conclusions:
Hierarchically organized board-and-staff-run COSPs combined with CMHA service may be less helpful than CMHA service alone. (Psychiatric Services 62:915–921, 2011)
In the past 25 years, the number of consumer-operated service programs (COSPs) for persons with serious mental illness has greatly increased. COSPs have become a major component of the mental health system and have been recognized as effective in fostering recovery (1). Government agencies (such as the Community Support Program of the Center for Mental Health Services of the Substance Abuse and Mental Health Services Administration) and advocacy organizations (such as the National Empowerment Center and the National Coalition of Mental Health Consumer-Survivor Organizations) promote a wide range of COSPs, including drop-in centers, clubhouses, independent living centers, case management services, employment agencies, and supported housing (2–8). When surveyed, almost half of a national sample of community mental health agencies (CMHAs) reported that they have referred clients to COSPs. CMHAs at which people with serious mental illness made up more than 50% of the caseload were almost 2.5 times more likely to make referrals; those where the CMHAs maintained a formal collaboration with a COSP were 7.5 times more likely to refer (9). These findings validate the growing and systematically supported role of COSPs in the provision of service to people with serious mental illness. Given their sanction as a future priority for funding, for example in California (10), COSPs are likely to play an increasing role in working with CMHAs to develop and implement treatment plans for individuals seeking services in public mental health systems. The objective of this study was to determine the effectiveness of such combined services for people with serious mental illness.
COSPs have certain key elements or “common ingredients” of program structure: Johnsen and colleagues (11) listed client involvement in hiring decisions, governance, and budget control; Mowbray and colleagues (12) cited client control in determining policies and procedures and a nonhierarchical program structure. However, COSPs have faced challenges in adhering to nonhierarchical structures. The COSP literature distinguishes two types of consumer-run organizations on the basis of their governance structures: the participatory, democratic self-help agency and the board-and-staff-run program (13,14). Self-help agency participant democracies give consumer-members a direct voice in major organizational decisions through participatory processes such as community meetings. Board-and-staff-run COSPs, like other COSPs, are primarily nonprofit organizations, with a director who is a consumer, at least 50% consumer representation on the board of directors, and consumer authority to hire and fire professionals employed within the organization (15). In the board-and-staff-run COSP, however, this authority resides in hierarchically structured consumer leadership rather than collective decision-making processes. Leaders tend to be selected for their vision and ability to get things done (13). Given these characteristics, board-and-staff-run COSPs have been easier to develop because they do not require the creation and time-intensive implementation of a collective governance process. They are also more widespread and are thus more likely to be the type of organizational form chosen for future expansion of COSPs (16); their challenge is to ensure that leadership is accountable to the membership (16–18).
Although enthusiasm for COSPs is widespread, few consumer-managed interventions have been subjected to controlled trials (19–21). There also is limited understanding of how differential forms of COSPs contribute to achieving recovery objectives. One randomized controlled trial has demonstrated the effectiveness of COSPs organized as self-help agency participant democracies working in concert with CMHAs to promote recovery objectives (22). In this study, using similar methodology, we considered the effectiveness of board-and-staff-run COSPs in working with CMHAs to achieve recovery objectives.
Methods
Study setting and participants
The study design is a two-group randomized clinical trial, with a board-and-staff-run COSP and a CMHA serving a midsize urban area. The COSP we used includes peer support groups, material resources, drop-in socialization, and direct services. It provides help with money management and payeeship services, counseling, case management, peer counseling, assistance in obtaining survival resources (such as food, shelter, and clothing), and provision of information or referral. Its drop-in center program is open six days a week approximately eight hours a day with an average daily attendance of 50 individuals. It promotes mutual support between members. All members participate in a community meeting where program ideas are discussed and decided by vote. Major organizational decisions, such as budget and personnel actions, are delegated to staff, administrators, and the organization's governing board. A former mental health care consumer serves as the COSP director, and the governing board's membership comprises a majority of consumers; this leadership has control over personnel and budgetary decisions. Its organizational model is that of a “top-down” board-and-staff-run agency (13). The high degree of staff control distinguishes this program from five other COSPs in the region (23).
The CMHA was a county mental health organization that provides outpatient mental health services for people with mental illness. These services include assessment, medication review, individual and group therapy, case management, and referral. In addition to professional providers, several consumer case managers staff the CMHA, an increasingly common practice in CMHA service provision.
Study design
New CMHA clients, seen at a time when a researcher was available (N=189) between mid-1999 and 2001, were invited to enroll in the study; 162 (86%) agreed to participate. No significant differences in gender, race-ethnicity, and housing status were found between clients who agreed to participate and those who refused.
The CMHA and COSP programs were chosen for their proximity to each other (distance between the sites was 2.5 miles), which facilitated successful referrals and utilization comparisons. After entry into the CMHA and agreement to participate in the study, clients were randomly assigned to continued CMHA outpatient treatment or to a combination of COSP and CMHA services at a ratio of approximately 2:3, respectively, favoring the referral condition of COSP-CMHA combined services). The proportion assigned to CMHA-only service in the randomization process was used to ensure that an appropriate number of referrals were completed. This did not prove to be a problem.
Random assignment of clients accepted for CMHA services by a clinician occurred at the CMHA immediately after clients consented to participate in the study. After the decision to admit the client and the client's agreement to participate in the study, the clinician opened an envelope that contained the client's service assignment. The clinician told the client that he or she “would be assigned to a provider at the clinic and asked to attend a consumer-operated drop-in program” or simply that he or she would be “assigned to a provider.” Thus the group referred to COSP received both CMHA and COSP services, enabling a comparison of outcomes from receipt of both services with those from CMHA-only service.
Institutional review board approval was obtained for all study procedures, and all study participants provided informed consent.
Assessment and measurement
Interview schedules included questions on demographic characteristics and recovery indicators as well as the Diagnostic Interview Schedule for DSM-IV (DIS-IV) (24). These were pretested with a sample of 310 long-term users of self-help agencies in northern California (25) and with 30 CMHA clients. Interviews were conducted by former mental health clients and professionals trained by the Center for Self Help Research in Berkeley, California.
Participants were assessed on five outcome measures. The Personal Empowerment Scale (26) and the Self-Efficacy Scale (27), respectively, measure individuals' control of and efficacy in their everyday lives. Individual social integration was assessed with the Independent Social Integration Scale (ISIS) (28,29), which measures five dimensions: social presence, access, participation, production, and consumption behaviors. Psychological functioning was assessed with the Brief Psychiatric Rating Scale (BPRS) (30). The Hopelessness Scale (31) assessed for a lack of hope. Higher scores on all scales indicated a higher degree of the measured characteristic. All scales possessed high internal consistency (α=.83–.95). Stability scores, measured across a six-month period, were in the moderate range (r=.48–.69). Each scale has independently established validity (25). BPRS interrater reliability was maintained throughout the study at a minimum of .80 with training film assessments.
Analyses
Analyses were completed with SPSS Version 18.0.0 (32). Frequencies and means were computed for the sample's descriptive characteristics. Differences in individual characteristics between the COSP-CMHA and CMHA-only samples' baseline characteristics observed in the follow-up samples were evaluated with chi square and analysis of variance tests. When between-condition differences were significant, the variable was selected for inclusion as a covariate control in the multivariate model evaluating the significance of outcome change attributable to the study condition.
Drop-in programs have individuals who attend with varying degrees of commitment; some individuals simply drop in for a cup of coffee, some to get out of the rain. Early work on such programs has emphasized the positive experience of “long-term users” of these services (25). To assess the experience of those fully participating in the service effort, we used SPSS's generalized linear model (GLM) [weighting option to weight the samples by the inverse of the probability of reaching eight months of service in the client's assigned service condition—that is, the number of people enrolled in the condition divided by the number who reached eight months of service. Data for those who did not reach eight months of service were given a weight of zero. This process gave greater importance to those completing the eight months of service and discounted those who did not. Results (see below) indicated virtually no difference in the probability of attaining the eight-month goal across conditions.
Using SPSS's GLM multivariate procedure, we evaluated the change in recovery-focused outcomes associated with service condition assignment over an eight-month period. The GLM multivariate enabled an analysis of covariance with dummy variable regression that demonstrated the impact of service condition while taking into account baseline status on each outcome measure and observed between-group differences. Change scores (eight-month outcome score minus baseline) for each of the five recovery-focused outcomes were the dependent measures. Service condition (combined service, coded 1, versus CMHA only, coded 0) was the fixed factor. Baseline scores on all five outcome measures and the demographic characteristics of gender, education (less than high school, coded 0; high school completion or some postsecondary, coded 1), employment (employed, coded 1; not employed, coded 0), and receipt of Supplemental Security Income (yes=1, no=0) were the model covariates. The model was run twice. First, only the baseline scores on the outcome measures were used as covariates along with service condition to predict change outcomes; second, we entered the additional demographic covariates and reran the model.
Results
Of the 162 consenting participants in the weighted sample at baseline, 139 (86%) continued eight months of service: 86 (86%) in the combined-service condition and 53 (85%) in the CMHA-only condition. No one in the latter group self-referred to the COSP during the study. The mean±SD number of visits per month to the COSP was 4.24±.43 (median=4; mode=4); to the CMHA it was 2.1±1.7 (median=1; mode=1). The sample's mean age was 37.0±9.8, 75 (54%) were male, 121 (87%) were Caucasian, and 57 (41%) had a DIS-IV-assessed diagnosis of schizophrenia or schizoaffective disorder. Demographic and diagnostic information is listed in Table 1.
The multivariate analyses indicated that the two service condition groups changed differently across time with and without controls for the demographic differences. The initial model, including the service condition plus the five baseline outcome scores as covariates (service condition Wilks' λ=.78, F=6.92, df=5 and 123, p<.001), indicated that this change was accounted for by two of the five recovery-focused outcomes: independent social integration (F=30.19, df=1 and 127, p<.001) and personal empowerment (F=4.32, df=1 and 127, p<.04). When the demographic controls were added to the final model, results remained significant (service condition Wilks' λ=.81, F=5.72, df=5 and 119, p<.001), and three of the recovery-focused outcomes significantly contributed to the change: independent social integration (F=16.94, df=1 and 123, p<.001), personal empowerment (F=7.98, df=1 and 123, p<.006), and self-efficacy (F=12.16, df= 1 and 123, p<.001) (Table 2). All such changes favored the CMHA-only service condition. Scores for participants in the CMHA-only condition increased for social integration (mean increase=20.34±5.36 out of a possible 261), personal empowerment (mean increase=8.21±1.02 out of a possible 57), and self-efficacy (mean increase=5.04±1.69 out of a possible 56). In the combined-service condition, however, there were notable decreases in social integration (−9.29±4.05) and self-efficacy (−2.89±1.28), and there was a modest increase in personal empowerment (4.34±.77). Symptoms, assessed with the BPRS, and hopelessness showed no difference in change between groups over the eight months.
The overall effect size for service condition, measured by partial eta squared, was 19% of the variance. The individual outcomes yielded a partial eta squared of 12% for independent social integration, 6% for personal empowerment, and 1% for self-efficacy. The overall effect size can be characterized as large (>14%); the effect size for independent social integration and personal empowerment was medium (6% to 14%), and it was small for self-efficacy (<6%) (33,34).
The difference in average improvement between the CMHA-only and the combined-service conditions (that is, the absolute-risk reduction) was 15% for social integration, 7% for personal empowerment, and 14% for self-efficacy. The “number needed to harm,” by making a referral for each outcome, was, respectively, 6.6 for social integration, 14.7 for personal empowerment, and 7.1 for self-efficacy.
Discussion
Adding consumer-run helping efforts to CMHA services usually produces better outcomes. Our findings indicate, however, that joint board-and-staff-run COSP and CMHA services underperformed CMHA services alone. Independent social integration, personal empowerment, and self-efficacy improved to a greater extent in the CMHA-only condition than in the combined-service condition. On average, taking the group demographic differences into account, positive change in the CMHA-only sample was approximately three times that of the COSP-CMHA sample for social integration and self-efficacy, and twice that for personal empowerment.
These findings are best understood with reference to a similar trial (using the same methodology and outcome measures) in which the combined efforts of democratically run consumer self-help agencies and CMHAs substantially outperformed the CMHA-only condition (22). As noted earlier, self-help agencies and the board-and-staff-run COSPs differ. The former allow members a direct voice in major organizational decisions; the latter assign such decisions to staff. Was the CMHA-only condition more successful in this trial of a combined board-and-staff-run COSP and CMHA effort because the CMHA in this trial was superior to the CMHAs in the trial with self-help agencies? Unfortunately no; the participants in the CMHA-only condition in the self-help agencies trial had independent social integration, personal empowerment, and self-efficacy outcomes that were not significantly different from those in the CMHA-only condition in this board-and-staff-run trial (22). [An appendix providing details on those results is available in an online supplement to this article at ps.psychiatryonline.org.] Given this information, the performance of participants in the combined condition cannot be attributed to superior performance by the CMHA in this study. In the study of self-help agencies the referral group did better than the CMHA-only group, whereas in this study of board-and-staff-run COSPs the referred group did worse. In both studies CMHA-only participants performed similarly, falling between the high-performing referrals in the former and the low-performing referrals in the latter. The fact that the CMHA in this study had consumers working alongside professionals as case managers apparently made no difference in the performance of the CMHA participants in this trial.
Although there were baseline differences in the referral and CMHA-only sample characteristics in this study, these differences were taken into account in the analyses and thus are not likely to explain the underperformance of the joint effort herein. Although it is difficult to attribute the referral population's poorer experience to a particular aspect of the COSP, its top-down decision-making procedures stand out as a probable contributing factor. The founding principles of consumer-led organizations emphasize the central importance of bottom-up as opposed to top-down administration—in the consumer movement's terms, democracy-controlled decision making among self-help participants versus board-and-staff-run decision making in the COSP (17). From the outset of the movement, consumer leadership expressed concern about the possible failure of the latter to achieve the outcome potentials presumed inherent in the former. The consumer-led organization in this study was described by 49 consumer self-referrals as emphasizing organizational efficiency and staff control to the exclusion of membership participation in agency decision making, with the consequence of lower levels of membership organizational empowerment (23).
Historically, mental health consumers developed their own programs as alternatives to disempowering, professionally run services that limited participant self-determination. Such programs were based on the principle that participants can help themselves and each other through peer support, power sharing, and member control of services (17). Through these empowering processes, members were to regain hope, self-esteem, and self-confidence lost through stigmatization as persons labeled “mentally ill” (2,35–37). Finding oneself in a COSP that fails to organizationally empower may compromise recovery outcomes. McLean (38) documented the unfulfilled promise of a COSP drop-in center embedded in the community mental health system. Her findings associated the combined-services approach with the compromise of the key COSP principles of consumer participation in empowering decision making and the sacrifice of these goals to bureaucratic and funding pressures.
Early research indicated that “institutional neurosis” developed as an iatrogenic effect associated with the hierarchical structures of psychiatric hospitals, documenting the link between organizational structure and behavioral outcomes (39). This link has been made in community-based residential care homes with documentation of the behavioral impacts of management- versus client-centered organizations (40). The results herein seem to demonstrate the risks of hierarchical structures in a community-based organization.
This study has its limitations. It involved a single board-and-staff-run COSP. Whether the hierarchical structure that characterized this organization characterizes other board-and-staff-run COSPs is an empirical question yet to be addressed. This study represented only one case in a large number of possible organizational referral arrangements between board-and-staff-run COSPs and CMHAs. The study sample included a high proportion of individuals with major depression and none who were homeless. Although these sample characteristics were present in both study conditions, they may be less common in other settings. Statistically, the results are not generalizable beyond the participating programs. Yet the findings of this study are consistent with the consumer literature on the importance of organizational empowerment in the success of consumer-run program efforts, and they mirror the type of negative behavioral consequences seen in other disempowering mental health organizations. Further, the importance of any single-site randomized clinical trial should not be minimized, particularly when the findings represent a possible distinction between two interventions: a very helpful consumer-operated, democratically run self-help service and what appears to be an unhelpful consumer-operated board-and-staff-run service. These interventions now seem to be conflated into a single approach called consumer-operated services. Like all such findings, the findings of this study need confirmation through further research.
Conclusions
COSPs have made positive contributions to the alleviation of past stigmatization and disempowerment. Yet without a clearly defined process at the COSP for empowering members, referral from a CMHA to a COSP should be supported with caution, in that it may be less helpful than CMHA services alone.
Acknowledgments and disclosures
This study was supported by research grant RO1-MH37310 from the National Institute of Mental Health and by the Mack Center on Mental Health and Social Conflict.
The authors report no competing interests.
References
1.
Achieving the Promise: Transforming Mental Health Care in America. Pub no SMA-03-3832. Rockville, Md, Department of Health and Human Services, President's New Freedom Commission on Mental Health, 2003
2.
Chamberlin J: On Our Own: Patient Controlled Alternatives to the Mental Health System. New York, McGraw-Hill, 1978
3.
Deegan PE: The Independent Living Movement and people with psychiatric disabilities: taking back control over our own lives. Psychosocial Rehabilitation Journal 15:3–19, 1992
4.
Felton BJ: Defining location in the mental health system: a case study of a consumer-run agency. American Journal of Community Psychology 36:373–386, 2005
5.
Herman SE, Onaga E, Pernice-Duca F, et al.: Sense of community in the clubhouse programs: member and staff concepts. American Journal of Community Psychology 36:343–356, 2005
6.
Humphreys K: Special section on self-help/mutual aid initiatives by people with psychiatric disabilities. Community Psychologist 29:9–25, 1996
7.
Mowbray CT, Chamberlain P, Jennings M, et al.: Consumer-run mental health services: results from five demonstration projects. Community Mental Health Journal 24:151–156, 1988
8.
Nikkel RE, Smith GD, Edwards D: A consumer operated case management project. Hospital and Community Psychiatry 43:577–579, 1992
9.
Hardiman ER: Referral to consumer-run programs by mental health providers: a national survey. Community Mental Health Journal 43:197–210, 2007
10.
California Mental Health Services Act, 2005. Available at www.dmh.ca.gov/prop_63/mhsa/docs/Mental_Health_Services_Act_Full_Text.pdf
11.
Johnsen M, Teague G, Herr EM: Common ingredients as a fidelity measure; in On Our Own, Together. Edited by, Clay S. Nashville, Tenn, Vanderbilt University Press, 2005
12.
Mowbray CT, Bybee D, Holter M, et al.: Validation of a fidelity rating I instrument for consumer-operated services. American Journal of Evaluation 27:9–27, 2006
13.
Budd S: Governing structures; in Reaching Across, Mental Health Clients Helping Each Other. Edited by, Zinman S, Harp HT, Budd S. Riverside, California Network of Mental Health Clients, 1987
14.
Chamberlin J: Direct democracy as a form of program governance; in Reaching Across II: Maintaining Our Roots/The Challenge of Growth. Edited by, Harp HT, Zinman S. Sacramento, California Network of Mental Health Clients, 1994
15.
Goldstrom I, Campbell J, Rogers J, et al.: National estimates for mental health mutual support groups, self-help organizations, and consumer-operated services. Administration and Policy in Mental Health and Mental Health Services Research 33:92–103, 2006
16.
Harp H, Zinman S: Maintaining our roots, the challenge for self-help development; in Reaching Across II: Maintaining Our Roots/The Challenge of Growth. Sacramento, California Network of Mental Health Clients, 1994
17.
Zinman S: Issues of power; in Reaching Across, Mental Health Clients Helping Each Other. Edited by, Zinman S, Harp HT, Budd S. Riverside, California Network of Mental Health Clients, 1987
18.
Chamberlin J: The ex-patient's movement: where we've been and where we're going. Journal of Mind and Behavior 11:323–336, 1990
19.
Davidson L, Chinman M, Kloos B, et al.: Peer support among individuals with severe mental illness. Clinical Psychology: Science and Practice 6:165–187, 1999
20.
Kyrouz EM, Humphreys K: Do psychiatrically disabled people benefit from participation in self-help/mutual aid organizations? A research review. Community Psychologist 29:21–25, 1996
21.
Rogers ES, Teague GB, Lichenstein C, et al.: Effects of participation in consumer-operated service programs on both personal and organizationally mediated empowerment: results of multisite study. Journal of Rehabilitation Research and Development 44:785–800, 2007
22.
Segal SP, Silverman C, Temkin T: Self-help and community mental health agency outcomes: a recovery-focused randomized trial. Psychiatric Services 61:905–910, 2010
23.
Segal SP, Silverman C, Temkin T: Are All Consumer-Operated Service Programs Empowering Self-Help Agencies? Berkeley, University of California, Mack Center on Mental Health and Social Conflict, 2011
24.
Robins LN, Cottler LB, Bucholz KK, et al.: Diagnostic Interview Schedule for the DSM-IV (DIS-IV). St Louis, Mo, Washington University in St Louis, 1999
25.
Segal SP, Silverman C, Temkin T: Characteristics and service use of long-term members of self-help agencies for mental health clients. Psychiatric Services 46:269–274, 1995
26.
Segal SP, Silverman C, Temkin T: Measuring empowerment in client-run self-help agencies. Community Mental Health Journal 31:215–227, 1995
27.
Bandura A: Self-efficacy: toward a unifying theory of behavioral change. Psychological Review 84:191–215, 1977
28.
Segal SP, Aviram U: The Mentally Ill in Community-Based Sheltered Care. New York, Wiley, 1978
29.
Segal SP, Kotler PL: Personal outcomes and sheltered care residence: ten years later. American Journal of Orthopsychiatry 63:80–91, 1993
30.
Overall J, Gorham D: The Brief Psychiatric Rating Scale. Psychological Reports 10:799–812, 1962
31.
Zimmerman MA: Toward a theory of learned hopefulness: a structural model analysis of participation and empowerment. Journal of Research in Personality 24:71–86, 1990
32.
SPSS Version 18.0.0: PASW Statistics. Chicago, SPSS, Inc, 2008. Available at www-01.ibm.com/software/analytics/spss/?S_CMP=ba_hp
33.
Cohen J: Statistical Power Analysis. Hillsdale, NJ, Erlbaum, 1988
34.
Pallant JF: SPSS Survival Manual: A Step by Step Guide to Data Analysis Using SPSS for Windows (Version 15), 3rd ed, rev. New York, Allen and Unwin, 2007
35.
Leete E: The role of the consumer movement and persons with mental illness. Presented at the 12th Mary Switzer Memorial Seminar in Rehabilitation. Washington, DC, June 15–16, 1988
36.
Kaufmann C, Freund P, Wilson J: Self-help in the mental health system: a model for consumer-provider collaboration. Psychosocial Rehabilitation Journal 13:5–21, 1989
37.
Zinman S, Harp HT, Budd S (eds): Reaching Across, Mental Health Clients Helping Each Other. Riverside, California Network of Mental Health Clients, 1987
38.
McLean A: Empowerment and the psychiatric consumer/ex-patient movement in the United States: contradictions, crisis and change. Social Science and Medicine 40:1053–1071, 1995
39.
Barton R: Institutional Neurosis. Bristol, United Kingdom, Wright, 1976
40.
Segal SP, Moyles EW: Management style and institutional dependency in sheltered care. Social Psychiatry 14:159–165, 1979
Figures and Tables
Information & Authors
Information
Published In
Cover: The South Gorge, Appeldore, Isles of Shoals, by Childe Hassam. Oil on canvas, 22¼ × 18 inches. Collection of the Newark Museum, Newark, New Jersey. Photo credit: the Newark Museum/Art Resource, New York.
History
Published online: 1 August 2011
Published in print: August 2011
Authors
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBGet Access
Login options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).