Implementing Evidence-Based Practices for Persons With Severe Mental Illnesses
Abstract
Extensive empirical research, summarized in several reviews and codified in practice guidelines, recommendations, and algorithms, demonstrates that several pharmacological and psychosocial interventions are effective in improving the lives of persons with severe mental illnesses. Yet the practices validated by research are not widely offered in routine mental health practice settings. As part of an effort to promote the implementation of evidence-based practice, the authors summarize perspectives on how best to change and sustain effective practice from the research literature and from the experiences of administrators, clinicians, family advocates, and services researchers. They describe an implementation plan for evidence-based practices based on the use of toolkits to promote the consistent delivery of such practices. The toolkits will include integrated written material, Web-based resources, training experiences, and consultation opportunities. Special materials will address the concerns of mental health authorities (funders), administrators of provider organizations, clinicians, and consumers and their families.
Practices that have been demonstrated to be effective by clinical services research could improve the lives of many people if they were widely adopted in routine health care settings. The desire to promote evidence-based practice has led to a proliferation of practice guidelines throughout medicine (1,2). As Rogers noted (3), government and professional organizations formulate guidelines "to help translate the scientific literature into concise statements intended to change practice." However, studies of the impact of practice guidelines suggest that publication and distribution of guidelines is not enough to change the practice of clinicians (2,4,5,6).
In the mental health field, several groups have published recommendations, guidelines, consensus statements, and treatment algorithms that are relevant to the care of adults with severe mental illnesses (7,8,9,10,11,12,13,14,15). Although healthy debates continue about the types and levels of research evidence, the efforts of these groups highlight the fact that much is known about how to help persons living with severe mental illnesses. For example, research strongly supports the use of specific medications prescribed in specific ways as well as the use of psychosocial interventions such as supported employment, various approaches to illness self-management, family psychoeducation, case management based on the principles of assertive community treatment, and substance abuse treatment that is integrated with mental health treatment (16).
More significant than disagreements about levels of evidence is the fact that the practices supported by research are not widely offered in routine practice settings. In the most extensive demonstration of this problem, the Schizophrenia Patient Outcome Research Team (PORT) showed that patients with a diagnosis of schizophrenia in two state mental health systems were highly unlikely to receive effective services (17). Even simple medication practices meet standards of effectiveness about half or less than half of the time. A minority of patients—often as few as 10 percent—receive psychosocial interventions supported by effectiveness research.
Evidence from other sources supports the PORT findings. For example, Anderson and Adams (18) noted that family psychoeducation is rarely available in routine practice settings, and Tashjian and associates (19) reported that fewer than 5 percent of persons with severe mental illnesses receive supported employment services.
Thus a critical challenge for the mental health field is to facilitate the widespread adoption of research-based practices in routine mental health care settings so that persons with severe mental illnesses can benefit from services that have been shown to work (20). In this paper we review the essential ingredients of an implementation plan for evidence-based practice.
The knowledge base
This paper is part of a series of reports that stem from the Implementing Evidence-Based Practices for Severe Mental Illness Project, which is sponsored by the Robert Wood Johnson Foundation, the Center for Mental Health Services of the Substance Abuse and Mental Health Services Administration, the National Alliance for the Mentally Ill (NAMI), and state and local mental health organizations in New Hampshire, Maryland, and Ohio. The project aims to develop implementation toolkits for effective practices. Subsequent papers will describe the project in more detail and address the issue of what constitutes evidence.
To inform the design of the implementation toolkits, we assessed five complementary sources of knowledge: the literature on efforts to change practice in health care; the discussions the authors had with executive administrators from provider organizations in New Hampshire, Maryland, and Ohio during several days of meetings on implementing evidence-based practice; the findings of focus groups with frontline clinicians who serve people with severe mental illnesses in New Hampshire and Baltimore; the advocacy experiences of NAMI; and the observations of services researchers who have implemented a range of mental health services in demonstration progiams.
Research literature
Implementing a practice involves promoting change in the behavior of mental health care providers. Theorists suggest that behavior changes when intention to change is combined with the necessary skill and the absence of environmental constraint (21). Green's model (22,23) for promoting change prescribes three elements: predisposing or disseminating strategies, such as educational events or written material; enabling methods, such as practice guidelines and decision support; and reinforcing strategies, such as practice feedback mechanisms.
Research shows that education alone does not strongly influence the practice behaviors of health care providers (24,25,26). Additional efforts, such as increasing consumer demand for services, changing financial incentives and penalties, using administrative rules and regulations, and providing clinicians with ongoing supervision and feedback on practices, are also necessary (24,27,28). The more elements of the system of care that can be marshaled to support change and reduce resistance, the more likely practice improvements will occur. In other words, intensity of effort appears directly related to success in studies of practice change (29,30).
Furthermore, complicated changes, such as modifying the practice of a whole clinical team, require a greater intensity of effort than is needed to affect a relatively simple change, such as shifting a single prescription pattern (3,29). Guidelines are not self-implementing and must be contextualized to the actual processes of care. Sustained change requires a restructuring of the flow of the daily work so that routine procedures make it natural for the clinician to give care in the new way (31).
Katon and colleagues (32), for example, were able to improve primary care treatment of depression to conform to treatment guidelines developed by the Agency for Health Care Policy and Research by using an intensive, multifaceted program that included extensive physician education and restructuring of services. They found, however, that the improved care did not generalize to other depressed clinic patients seen outside the restructured intervention program, even when the patients were seen by the same physicians, and did not endure after the processes of care reverted to their previous form when the year-long intervention was complete (25).
Research also indicates that when mental health programs attempt to implement evidence-based practices, the quality of the implementation strongly influences outcomes (33,34,35,36). In other words, if two programs offer a practice of care that is known to be effective, the program that has the higher fidelity to the defined practice tends to produce superior clinical results. This finding suggests that efforts to promote evidence-based practice must include fidelity measures and self-correcting feedback mechanisms.
Implementation efforts are most effective when they address the specific needs, values, and concerns of the persons whose behavior the implementation aims to change (37,38). Specifically, administrative elements of an implementation plan must be tailored to mental health administrators, clinical training elements to clinicians, and consumer and family education to those groups.
Administrators' perspective
Mental health administrators influence the implementation of evidence-based practice by setting priorities for care and organizing the operational details of practice. During meetings with the researchers in New Hampshire and Baltimore, administrators from organizations that provide services to adults with severe mental illnesses strongly urged the development of organized resources to help programs offer evidence-based practices. Having access to clearly defined practices that are known to work and that are packaged for implementation would help administrators in their efforts to meet their responsibility to the communities they serve. High turnover of clinical staff can easily overwhelm the training resources of any one agency and underscores the need for focused, effective training materials that can be used repeatedly.
Administrators indicate that financial incentives and administrative rules and regulations must be aligned to support the implementation of evidence-based practices. The overall financing structure must be relatively stable and support the effort. To sustain evidence-based practice over time, ongoing funding must be competitive with the potential income from alternative clinical staff activities. Additional financial concerns of administrators are the cost of initial training and the cost of collecting clinical outcome data and measuring fidelity over time. Training must be efficient because the cost of training includes both the direct costs and, in fee-for-service settings, the opportunity costs of the training time. The processes for tracking clinical outcomes and practice fidelity need to be streamlined because data collection takes time and energy away from direct clinical service.
Clinicians' perspective
To understand the needs of clinicians who directly provide services to adults with severe mental illnesses, we conducted a series of seven focus groups with case managers, psychiatrists, vocational specialists, and community support program therapists from numerous practice sites in New Hampshire and Baltimore. The purpose was to ascertain what factors motivate them to change, how they learn a new practice, and what they perceive as barriers to change.
Clinicians indicated that, first, they must be convinced that the practice is worth learning; second, they need to learn a practice through observation, training, and reading; and, third, they benefit from efforts to reinforce the practice over time, such as regular supervision and feedback on activities.
Clinicians are generally not eager to change and must be convinced to adopt a new practice. In addition to research support, they are influenced by compelling vignettes, impressions of the practice seen in action, and a practice ideology, or theory, that resonates with their values and experiences as providers. Practices that can be learned and put into action quickly are more appealing than those that require intensive learning or dramatic system change.
Clinicians are particularly motivated to learn a new practice if they believe it will help them in a clinical area where they currently feel ineffective or if they feel there will be broad consumer demand for the practice. An opportunity to discuss rationale, theory, and concerns is helpful to clinicians as they evaluate a proposed practice.
Once convinced to adopt a practice, clinicians need practical instruction. At this stage, most clinicians indicate that seeing the practice in action has more of an impact than reading about it. They endorse traveling to a training site where the practice can be observed, being visited by a skilled trainer who can demonstrate the practice, or watching instructional videotapes. Written materials complement and expand understanding gained from observing the practice. Clinicians prefer practical workbooks with clear and simple aims, principles, and examples, rather than books. The workbook should provide a clear structure of practice with examples and challenging questions. Ideally the workbook should lay out the practice in brief, easy-to-read modules allowing a clinician to read a section and immediately try the intervention.
Clinicians cite the importance of supports to consolidate and reinforce new practices. Practice-specific supervision of their clinical work is essential. Supervision helps clinicians translate the theory into daily action. Supervision can be done by an expert at a distance over the phone or, preferably, by a skilled local program leader who can motivate others to learn. Strategically placed posters listing key practice principles or relevant practice slogans support the supervision and reinforce learning through repetition. Follow-up trainings and feedback on practice are also helpful. Clinicians suggest that internet resources, advanced seminars, and additional readings may all be helpful for people who have the basics and want to extend their understanding and skill.
Advocates' perspective
Advocacy can help systems move to research-based practice. For example, the National Alliance for the Mentally Ill (NAMI) has had success promoting the Program for Assertive Community Treatment (PACT). By focusing on the replication of PACT as a national priority, packaging the practice for implementation, engaging the media, coordinating NAMI state organization efforts, and communicating progress, NAMI has created a grassroots demand for PACT. Active NAMI PACT steering committees are working with providers to establish PACT in 19 states.
From the PACT dissemination effort, NAMI has learned the importance of designing communication to match the needs of the participants and reinforcing learning through repetition. NAMI created printed material to capture the attention of policy makers and administrators at the state and local levels, built information for family members into family education programs, and developed a detailed implementation manual for providers. NAMI promotes PACT at every opportunity, including media events, annual NAMI conventions, state conferences, and on the Web.
Observations of researchers
The final source of knowledge about practice implementation is the combined experience of the authors from implementing and studying medication algorithms and psychosocial interventions in routine practice settings. Our experiences make it clear that a core set of steps is required to successfully establish and maintain a desired practice. These include clearly voiced administrative support for change before training; initial clinical training using didactic methods, observation of practice, and written materials; ongoing weekly supervision by an expert, based on written principles and practices; follow-up visits by a program expert with feedback on implementation; and feedback on services and outcomes.
Model programs often dissipate after demonstration or research projects end. Practices that persist have funding and organizational structures to support the practice. These programs typically have an influential clinical or operational leader who ensures the continuity of the structures and training that maintain the practice (39,40,41,42,43).
Implementation toolkits
On the basis of the knowledge reviewed above, the team for the Implementing Evidence-Based Practices for Severe Mental Illness Project proposes a model for effecting change. The team will develop integrated written material, Web-based resources, training experiences, and consultation opportunities, packaged as implementation toolkits. The toolkits will be designed to promote the consistent delivery of effective services and will be developed for practices for which the evidence base is extensive and the consensus supporting the practice relatively high. The impact of the toolkits on practice will be carefully studied to learn more about how to translate research-based knowledge into practice in the care of adults with severe mental illnesses.
Clearly, practice implementation can easily fail. To succeed, the system of care must have adequate resources and be reasonably organized, and the efforts of multiple stakeholders must be aligned to support the practices. Stakeholders who play essential roles include mental health authorities (funders), who create administrative rules and financial incentives; administrators of mental health organizations, who set priorities and organize care; mental health clinicians, who provide direct care; and service consumers and their families, who create demand for and receive mental health care. The toolkits will address all of these stakeholders.
The first challenge is to predispose stakeholders to work on restructuring services. For each practice, materials will include a brief video that describes the evidence-based practice, reviews the scientific support, and provides testimony from clients and clinicians who have participated in the intervention. The video will link the research evidence for the practice with the personal experiences of people who have benefited from receiving the intervention, thereby making the practice come to life for administrators, clinicians, and consumers and their families.
Written material for each practice will also include a list of commonly asked questions and answers, a paper summarizing the scope and limits of the evidence supporting the practice, and a general paper articulating the rationale for adopting practices that are strongly supported by research. In addition to the materials, introductory training and consensus-building group facilitation will be offered to actively engage stakeholders in the change process.
Second, once the stakeholders are predisposed to change, the toolkits will enable them to make the desired changes. For administrators of mental health authorities, implementation toolkits will include a brief document specifying the funding structures and administrative rules that create effective incentives, administrative consultation, and links to other administrators who have found mechanisms to promote specific evidence-based practices. To support change efforts by administrators of provider organizations, implementation toolkits will include general recommendations for promoting change in health care settings; a brief document that lays out recommended practice processes, such as staffing, training, meeting structure, supervision, and fidelity monitoring; and administrative consultation to help organizations overcome obstacles to establishing the practice.
Implementation toolkits for clinicians will recommend initial training, including job shadowing at a site where the practice is well established. The primary ongoing training tool will be a practical workbook that articulates the aims of the practice and the practice principles and provides clear examples. Additional supports for clinicians will include extended practice-specific supervision by trainers to help clinical supervisors learn to apply practice principles to their specific clinical dilemmas, posters articulating the practice principles, and a Web site for the practice to link providers with other relevant research, training opportunities, and materials. For consumers and their families, a booklet will be developed that describes the practice, indicates reasonable expectations about services and outcomes, and refers people to organizations that advocate for practices, such as NAMI.
Third, once an evidence-based practice has been established, organized practice feedback will help stakeholders maintain and extend the gains. To provide system feedback, implementation toolkits will include fidelity scales for program self-assessment, simple outcome measures to track the effects of the practice, follow-up trainings either at the practice site or a training site, and a recommended process for review and revision of the implemented practice to further adapt it to site-specific circumstances and to continue to engage stakeholders in implementation.
Conclusions
Extensive empirical research demonstrates that several pharmacological and psychosocial interventions are effective in improving the lives of persons with severe mental illnesses. Despite this knowledge, there has been widespread failure to implement evidence-based practices in routine mental health settings. In this report, we have recommended ingredients of an implementation plan for evidence-based practices based on the perspectives of the research literature, administrators of provider organizations, clinicians, family advocates, and services researchers. The effectiveness of the proposed plan will be studied and the findings shared with the field.
Acknowledgments
This article was supported by contract 280-00-8049 from the Center for Mental Health Services, a grant from the Robert Wood Johnson Foundation, and grants MH-00839 and MH-56147 from the National Institute of Mental Health. The authors thank Paul B. Batalden, M.D., for his comments on an earlier draft of this article.
Editor's Note: This article by Dr. Torrey and his colleagues initiates a series of papers on evidence-based practices that is planned for monthly publication in 2001. These papers will focus on mental health practices for which there is substantial evidence of effectiveness and should therefore be routinely offered in clinical settings. Other articles in the series will address evidence-based practices for supported employment, for patients with dual diagnoses, for case management, for children's services, and for family psychoeducation, among other topics. Robert E. Drake, M.D., Ph.D. ([email protected]), and Howard H. Goldman, M.D., Ph.D. ([email protected]), are the series editors. Please contact them about submitting papers.
Footnote
Dr. Torrey is associate professor of psychiatry, Dr. Drake is professor of psychiatry, and Dr. Clark is associate professor of community and family medicine and psychiatry at Dartmouth Medical School in Hanover, New Hampshire. Dr. Dixon is associate professor of psychiatry at the University of Maryland in Baltimore. Dr. Burns is professor of medical psychology at Duke University Medical Center in Durham, North Carolina. Ms. Flynn is former executive director, National Alliance for the Mentally Ill in Arlington, Virginia. Dr. Rush is professor of psychiatry at the University of Texas Southwestern Medical Center at Dallas. Dr. Klatzker is chief executive officer of the Riverbend Community Mental Health Center in Concord, New Hampshire. Address correspondence to Dr. Torrey at the New Hampshire-Dartmouth Psychiatric Research Center, 2 Whipple Place, Lebanon, New Hampshire 03301 (e-mail, [email protected]).
References
1.
Audet AM, Greenfield S, Field M: Medical practice guidelines: current activities and future directions. Annals of Internal Medicine 113:709-714, 1990
2.
Lomas J: Words without action? The production, dissemination, and impact of consensus recommendations. Annual Review of Public Health 12:41-65, 1991
3.
Rogers EM: The challenge: lessons for guidelines from the diffusion of innovations. Journal on Quality Improvement 21:324-328, 1995
4.
Cabana MD, Rand CS, Powe NR, et al: Why don't physicians follow clinical practice guidelines? A framework for improvement. JAMA 282:1458-1465, 1999
5.
Kosecoff J, Kanouse DE, Rogers WH, et al: Effects of the National Institutes of Health consensus development program on physician practice. JAMA 258:2708-2713, 1987
6.
Lomas J, Anderson GM, Domnick-Pierre K, et al: Do practice guidelines guide practice? The effect of a consensus statement on the practice of physicians. New England Journal of Medicine 321:1306-1311, 1989
7.
American Psychiatric Association: Practice guidelines for the treatment of patients with bipolar disorder. American Journal of Psychiatry 151 (Dec suppl):1-36, 1994
8.
American Psychiatric Association: Practice guidelines for the treatment of patients with schizophrenia. American Journal of Psychiatry 154 (Apr suppl):1-63, 1997
9.
American Psychiatric Association: Practice guideline for the treatment of patients with major depressive disorder (revision). American Journal of Psychiatry 157 (Apr suppl):1-45, 2000
10.
Crismon ML, Trivedi MH, Pigott TA, et al: The Texas Medication Algorithm Project: report of the Texas consensus conference panel on medication treatment of major depressive disorder. Journal of Clinical Psychiatry 60:142-156, 1999
11.
Depression Guideline Panel: Clinical Practice Guideline, 5: Depression in Primary Care, vol 2. Treatment of Major Depression. AHCPR pub no 93-0551. Rockville, Md, Agency for Health Care Policy and Research, 1993
12.
Kahn DA, Carpenter D, Frances AJ, et al: The expert consensus guideline series: treatment of bipolar disorder. Journal of Clinical Psychiatry 57 (suppl 12A):1-86, 1996
13.
Lehman AF, Steinwachs DM, survey coinvestigators of the PORT project: Translating research into practice: the Schizophrenia Patient Outcomes Research Team (PORT) treatment recommendations. Schizophrenia Bulletin 24:1-10, 1998
14.
McEvoy JP, Scheifler PL, Frances A (eds): The expert consensus guideline series: treatment of schizophrenia 1999. Journal of Clinical Psychiatry 60(suppl 11):1-80, 1999
15.
Rush AJ, Rago WV, Crismon ML, et al: Medication treatment for the severely and persistently mentally ill: the Texas Medication Algorithm Project. Journal of Clinical Psychiatry 60:284-291, 1999
16.
Drake RE, Mueser KT, Torrey WC, et al: Evidence-based treatment of schizophrenia. Current Psychiatry Reports 2:393-397, 2000
17.
Lehman, AF, Steinwachs DM, survey coinvestigators of the PORT project: Patterns of usual care for schizophrenia: initial results from the Schizophrenia Patient Outcomes Research Team (PORT) client survey. Schizophrenia Bulletin 24:11-20, 1998
18.
Anderson J, Adams C: Family interventions in schizophrenia: an effective but underused treatment. British Medical Journal 313:505-506, 1996
19.
Tashjian M, Hayward B, Stoddard S, et al: Best Practice Study of Vocational Rehabilitation Services to Severely Mentally Ill Persons. Washington, DC, Policy Study Associates, 1989
20.
Mental Health: A Report of the Surgeon General. Rockville, Md, Center for Mental Health Services, 1999
21.
Fishbein M: Developing effective behavioral change interventions: some lessons learned from behavioral research, in Reviewing the Behavioral Science Knowledge Base on Technology Transfer. Edited by Backer TE, David SL, Soucy G. NIDA Research Monograph 155. Rockville Md, National Institute of Mental Health, 1995
22.
Green L, Kreuter M, Deeds S, et al: Health Education Planning: A Diagnostic Approach. Palo Alto, Calif, Mayfield, 1980
23.
Green L, Krueter M: Application of Precede/Proceed in Community Settings: Health Promotion Planning: An Educational and Environmental Approach. Mountain View, Calif, Mayfield, 1991
24.
Davis DA, Thomson MA, Oxman AD, et al: Changing physician performance: a systematic review of the effect of continuing medical education strategies. JAMA 274:700-705, 1995
25.
Lin EH, Katon WJ, Simon GE, et al: Achieving guidelines for the treatment of depression in primary care: is physician education enough? Medical Care 35:831-842, 1997
26.
Oxman AD, Thomson MA, Davis DA, et al: No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. Canadian Medical Association Journal 153:1423-1431, 1995
27.
Greco PJ, Eisenberg JM: Changing physician practices. New England Journal of Medicine 329:1271-1274, 1993
28.
Handley MR, Stuart ME, Kirz HL: An evidence-based approach to evaluating and improving clinical practice: implementing practice guidelines. HMO Practice 8:75-83, 1994
29.
Davis DA, Thomson MA, Oxman AD, et al: Evidence for the effectiveness of CME: a review of 50 randomized controlled trials. JAMA 268:1111-1117, 1992
30.
Schulberg HC, Katon W, Simon GE, et al: Treating major depression in primary care practice: an update of the Agency for Health Care Policy and Research practice guidelines. Archives of General Psychiatry 55:1121-1127, 1998
31.
Batalden PB, Stoltz PK: A framework for the continual improvement of health care: building and applying professional and improvement knowledge to test changes in daily work. Joint Commission Journal on Quality Improvement 19:424-445, 1993
32.
Katon W, Von Korff M, Lin EHB, et al: Collaborative management to achieve treatment guidelines: impact on depression in primary care. JAMA 273:1026-1031, 1995
33.
Drake RE, McHugo GJ, Becker DR, et al: The New Hampshire study of supported employment for people with severe mental illness. Journal of Consulting and Clinical Psychology 64:391-399, 1996
34.
Jerrel JM, Ridgely MS: Impact of robustness of program implementation on outcomes of clients in dual diagnosis programs. Psychiatric Services 50:109-112, 1999
35.
McDonnell J, Nofs D, Hardman M, et al: An analysis of the procedural components of supported employment programs associated with employment outcomes. Journal of Applied Behavioral Analysis 22:417-428, 1989
36.
McHugo GJ, Drake RE, Teague GB, et al: Fidelity to assertive community treatment and client outcomes in the New Hampshire dual disorders study. Psychiatric Services 50:818-824, 1999
37.
Grimshaw JM, Russell IT: Effect of clinical guidelines of medical practice: a systematic review of rigorous evaluations. Lancet 342:1317-1322, 1993
38.
Soumerai SB, Avorn J: Principles of educational outreach (academic detailing) to improve clinical decision making. JAMA 263:549-556, 1990
39.
Becker DR, Torrey WC, Toscano R, et al: Building recovery-oriented services: lessons from implementing IPS in community mental health centers. Psychiatric Rehabilitation Journal 22:51-54, 1998
40.
Burns BJ: A call for a mental health services research agenda for youth with serious emotional disturbance. Mental Health Services Research 1:5-20, 1999
41.
Liberman RP, Eckman TA: Dissemination of skills training modules to psychiatric facilities: overcoming obstacles to the utilization of a rehabilitation innovation. British Journal of Psychiatry 155(suppl 5):117-122, 1989
42.
Rush AJ, Crismon ML, Toprac MG, et al: Consensus guidelines in the treatment of major depressive disorder. Journal of Clinical Psychiatry 69(suppl 20):73-84, 1998
43.
Rush AJ, Crismon ML, Toprac MG, et al: Implementing guidelines and systems of care: experiences with the Texas Medication Algorithm Project (TMAP). Journal of Practical Psychiatry and Behavioral Health 5:75-86, 1999
Information & Authors
Information
Published In
History
Published online: 1 January 2001
Published in print: January 2001
Authors
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBGet Access
Login options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).