Implementing Evidence-Based Practices in Routine Mental Health Service Settings

Psychiatric Services' decision to dedicate 2001 to evidence-based practices rests on a series of research findings and philosophical commitments. First, a great deal is now known about efficacious and effective mental health interventions, which we refer to here as evidence-based practices. For example, numerous recent reviews of the research evidence identify a core set of interventions that help persons with severe mental illness attain better outcomes in terms of symptoms, functional status, and quality of life (2,3,4,5,6). The core set includes medications prescribed within specific parameters, training in illness self-management, assertive community treatment, family psychoeducation, supported employment, and integrated treatment for co-occurring substance use disorders (7). In upcoming issues, experts will review research evidence on outcomes, present current knowledge about barriers and strategies related to implementation, and discuss implications.

A second reason for the journal's focus on evidence-based practices is that despite extensive evidence and agreement on effective mental health practices for persons with severe mental illness, research also shows that routine mental health programs do not provide evidence-based practices to the great majority of clients with these illnesses (8). This finding was a major conclusion of the mental health report of the Surgeon General (6). In the most extensive demonstration of this problem, the Schizophrenia Patient Outcome Research Team showed that in two state mental health systems, clients with a diagnosis of schizophrenia were highly unlikely to receive effective services (9). For example, antipsychotic medications were often prescribed at dosages outside the effective range. A minority of clients—often as few as 10 percent—received evidence-based psychosocial services, such as family interventions. Findings from other sources suggest a dearth of evidence-based practices in routine mental health settings (10).

Third, research indicates that offering a service that resembles an evidence-based practice is not sufficient; adherence to specific programmatic standards, often referred to as fidelity of implementation, is necessary to produce expected outcomes (11,12,13). In other words, if two programs offer a practice of care that is known to be effective, the program with higher fidelity to the defined practice model tends to produce superior outcomes. This critical finding, which contradicts the conventional wisdom that model programs do not transfer and need to be modified extensively to fit local circumstances, suggests that implementation guidelines and toolkits should begin to incorporate manuals and fidelity measures (14).

Fourth, mental health services for persons with severe mental illness should reflect the goals of consumers. People with severe mental illness, like people with other long-term illnesses, want to pursue normal, functional, satisfying lives to the greatest extent possible (15,16). Mental health services therefore should not focus exclusively on traditional outcomes of treatment compliance and prevention of relapses and rehospitalizations. The new paradigm emphasizes helping people attain outcomes such as independence, employment, satisfying relationships, and good quality of life.

Fifth, given that mental health resources are limited, persons with severe mental illness have a right to have access to interventions that are known to be effective and that are delivered in a manner faithful to or consistent with current understandings of the interventions' active ingredients. In other words, to the extent that evidence-based practices exist, they should be the bedrock, the minimum of acceptable offerings, in all mental health settings that provide services for persons with severe mental illness. Additional services may enhance the service offering and may prove to be effective over time, but the basic offering of evidence-based practices should not be displaced by interventions of unknown or lesser effectiveness. Researchers, state mental health directors, consumers, and families support this commitment and have endorsed the urgent need for dissemination and implementation of evidence-based practices (17).

Finally, evidence-based practices do not provide the answers for all persons with mental illness, all outcomes, or all settings. Below we discuss several caveats as well as the judicious and informed use of evidence-based practices. Accurate information about effective practices, including a clear understanding of the limitations of current research and the need for further evidence in many areas, can improve services and outcomes. However, to produce positive changes, the research evidence on effective services and the implementation procedures must be available to all stakeholders in the service system (1).

Evidence-based practices are interventions for which there is consistent scientific evidence showing that they improve client outcomes. For example, research shows that using antipsychotic medications within specific dosage ranges and providing education and skill training for family caregivers over several months prevents or delays relapses of schizophrenia (4). The requirements for scientific evidence used by different groups sometimes vary, but in general the highest standard is several randomized clinical trials comparing the practice to alternative practices or to no intervention. When the separate trials are considered together, such as through a meta-analysis, the evidence supports the superiority of the evidence-based practice over the alternatives, including no intervention.

In some situations quasi-experimental studies, with comparison groups that are not assigned by randomization, constitute the best available evidence and consistently support a specific evidence-based practice. Rather than rigid decision rules, which inevitably are more appropriate for some types of interventions than others, we encourage panels of research scientists to review the available controlled studies and to make explicit their criteria for inclusion, the studies reviewed, the review procedures, and the conclusions so that others can examine the same evidence and reasoning.

Open clinical trials, which lack independent comparison groups, provide a significantly lower level of research evidence and are generally not considered to provide sufficiently strong scientific evidence. The lowest level of evidence, which should not be considered research evidence, consists of clinical observations collected as expert opinion. Best practices based on clinical opinions or open clinical trials do not constitute evidence-based practices, because they are not research based, are fraught with potential for error, and are often contradicted by later findings of controlled research.

Some groups, such as the Agency for Healthcare Research and Quality, have identified levels of scientific evidence, which they use to score evidence-based practices. The various practice guidelines developed in the 1990s by the agency—then known as the Agency for Health Care Policy and Research (18)—exemplify this approach by using three levels of evidence: level A refers to good research-based evidence, with some expert opinion. Level B indicates fair research-based evidence, with substantial expert opinion, to support the recommendation. Level C denotes a recommendation based primarily on expert opinion, with minimal research-based evidence.

Although the term evidence-based practice is sometimes used to refer to guidelines that are not based on research, true evidence-based practices are by definition grounded in consistent research evidence that is sufficiently specific to permit the assessment of the quality of the practices rendered as well as the outcomes. For example, supported employment is sufficiently well defined and standardized so that its fidelity can be measured and it can be differentiated from other approaches to vocational rehabilitation (14).

Other guidelines, such as those developed by the American Psychiatric Association (19), are based on a mixture of randomized clinical trials and expert consensus and often lack sufficient specificity to provide tools for judging the quality of interventions. There are also consensus guidelines, such as those developed by the triuniversity consortium (20), that systematically define desired practices through an iterative consensus process among experts. The advantage of the expert consensus approach is that it permits development of guidelines about practices for which no systematic research evidence exists. The major disadvantage is that the expert opinions may reflect current biases in the field rather than demonstrated effectiveness. At the far end of the continuum are guidelines that are written primarily to serve some administrative function and that are supported neither by research evidence nor by expert consensus.

Treatment decisions are typically complex, involving multiple decision steps that depend on the patient's response to treatment at each step. This complexity has led to the development of treatment algorithms that map out a series of decision points based on responses to the previous steps. For example, an algorithm for pharmacologic treatment of a person experiencing acute symptoms of schizophrenia may begin with initiation of an antipsychotic medication within a specific dosage range. If the person fails to respond adequately to the treatment, the algorithm may specify a second option—for example, switching to another class of antipsychotic agent. Failure to respond to the second treatment would lead to a third decision point—for example, initiation of clozapine treatment. Such algorithms may be particularly useful in guiding clinicians through a complex series of treatment decisions.

As with guidelines, algorithms may have varying levels of scientific evidence to support them. A current challenge in algorithm development is that the scientific evidence supporting the successive steps quickly becomes quite thin. Hence even the most evidence-based algorithms typically begin with steps supported by multiple clinical trials and evolve into steps defined through expert consensus. An excellent example of treatment algorithms has been developed in the Texas Medication Algorithm Project (5).

A number of questions arise in relation to the recent emphasis on evidence-based practices. For example, what is the role of ethical values in shaping practices? What should be done when there are different levels of evidence or changes in evidence? What are the limits of evidence? What should be done in clinical situations for which there is no scientific evidence? We briefly address each of these points below and will do so in greater detail in upcoming issues of the journal.

For each area of evidence-based practices, implementation in routine mental health practice settings is complex and difficult (1). Issues of organizational structure and commitment, resource development, and clarity of roles and responsibilities must be addressed before training can be effective (28). Service boundaries are often involved as well. For example, supported employment involves the interface between clinical and rehabilitative services, and dual diagnosis services highlight differences between the mental health and substance abuse treatment systems. Upcoming articles in Psychiatric Services will address implementation barriers and strategies as well as the specific evidence, including boundaries and extensions, for each of several evidence-based practices. Emphasis on evidence-based practices also has implications for public policy, education, research, medical information systems, managed care, liability, and many other topics.

Dr. Drake and Dr. Mueser are professors of psychiatry and Dr. Torrey is associate professor of psychiatry at the New Hampshire-Dartmouth Psychiatric Research Center, 2 Whipple Place, Lebanon, New Hampshire 03766 ( [email protected] ). Dr. Leff is assistant professor of psychiatry at Harvard Medical School and at the Cambridge Hospital in Cambridge, Massachusetts. Dr. Goldman and Dr. Lehman are professors of psychiatry and Dr. Dixon is associate professor of psychiatry at the University of Maryland School of Medicine in Baltimore.