Psychopharmacology: The Need for Effectiveness Trials to Inform Evidence-Based Psychiatric Practice

Although efficacy trials often compare a single active treatment to a placebo, effectiveness trials reflect real-world clinical situations by comparing two or more active treatments, which may include nonpharmacologic alternatives. For example, in the ongoing Sequenced Treatment Alternatives to Relieve Depression (STAR*D) trial for nonpsychotic major depressive disorder, patients without a satisfactory response to citalopram are randomly assigned to four switch options (sertraline, buproprion, venlafaxine, and cognitive therapy) or three augmentation strategies (buproprion, buspirone, and cognitive therapy) (5). The study algorithm leads those who do not show significant improvement through a sequence of subsequent relevant switch or augmentation strategies. The STAR*D trial will thus address the effectiveness of commonly used treatment strategies that currently lack an evidence base. Similarly, as part of the Systematic Treatment Enhancement Program for Bipolar Disorder (STEP-BD), several effectiveness trials will be conducted to compare commonly used but unproven treatments for bipolar depression and relapse prevention (6). In the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) project, a series of blinded and open-label randomized trials of active treatments will help answer questions about the comparability of commonly used medications for schizophrenia and strategies for addressing treatment resistance (7).

Whereas efficacy trials carefully select highly motivated, homogeneous populations without significant psychiatric or medical comorbid conditions, patients from real-world settings are frequently ambivalent about treatment, come from a variety of ethnic and socioeconomic backgrounds, and have significant comorbid conditions. For example, compared with patients with Alzheimer's disease in community samples, similar patients enrolled in clinical trials are better educated, wealthier, younger, less medically ill, and less behaviorally disturbed; they deteriorate more slowly and experience lower mortality rates (8). Only 4 to 8 percent of the patients with Alzheimer's who were listed in a large clinical database would have met eligibility criteria for enrollment in two clinical trials of cognitive enhancers (8).

Similarly, Keitner and colleagues (9) evaluated the generalizability of recent efficacy trials of antidepressants for major depressive disorder and determined that 86 percent of study applicants were ineligible for the trials: 17 percent had bipolar disorder, 16 percent abused substances, 14 percent had mild depression, 13 percent had a medical contraindication, and 12 percent were using other psychotropics that were not permitted in the trial. These trials thus do not address the common clinical presentations seen in practice.

In contrast, Partners in Care, a trial of quality improvement strategies for depression in managed primary care, screened patients in waiting rooms to recruit a diverse patient population (10). Of the 1,356 patients enrolled, 40 percent had subthreshold depression commonly found in primary care, 43 percent had comorbid anxiety disorders, 33 percent had problematic alcohol or drug use, and 79 percent had one or more chronic medical conditions. Additionally, 43 percent of study participants were from a racial or ethnic minority group.

Effectiveness trials often take place in real-world settings and use usual providers. Efficacy trials are often conducted in specialty academic or commercial research settings, even though epidemiologic evidence suggests that most patients with depression and anxiety who receive treatment do so in primary care (11,12). Participants in efficacy trials also receive costly, state-of-the-art interventions and an intensity of clinical care not frequently available to patients in usual-care settings. For example, even participants who receive a placebo are scheduled for weekly follow-up visits that involve supportive interactions and the monitoring of active symptoms. Even the process of obtaining informed consent educates and engages participants to a greater extent than may occur in usual clinical settings. Providers of medication or psychotherapy are often experts in their field, potentially influencing the quality and outcomes of treatments delivered. Thus, although efficacy trials may represent "cookies baked by an expert chef according to a recipe," real-world treatments are similar to "cookies baked at home, loosely based on a recipe, often with some substitution of ingredients" (personal communication, Duan N, 2004).

Effectiveness trials may help us understand how to best implement treatments in real-world settings to optimize outcomes. Partners in Care (10) and Project Improving Mood: Promoting Access to Collaborative Treatment (IMPACT) (13), among others, implemented depression treatment in real-world primary care settings, using usual care as the comparison group, with usual providers delivering study treatments after receiving training by study experts. In Partners in Care a significant difference was seen between the two groups: 51 percent of patients in the intervention group compared with 40 percent of patients in the usual-care group received minimally adequate depression care (p<.001) (10). In Project IMPACT a significant difference was also seen between the two groups: 78 percent of patients in the intervention group compared with 54 percent of patients in the usual-care group received adequate depression care (p<.001) (13).

Finally, effectiveness trials tend to examine a broader range of outcomes than traditional efficacy trials. They frequently include outcomes that are highly relevant to patients, including not only symptom burden but also functioning, quality of life, satisfaction, and costs. Persons who make health care decisions often benefit from the inclusion of cost-effectiveness and cost-utility outcomes, both of which determine the costs of clinical gains or improvements in quality of life, respectively (3).

For example, costs may be determined for the impact of treatments in terms of quality-adjusted life years, a measure that takes into account both the quantity and quality of life generated by an intervention. As a standardized measure, the quality-adjusted life year (QALY) can be used to compare the relative merits of different treatments across disease conditions and populations. For example, in Partners in Care the estimated cost per QALY gained for Latinos was $6,100 or less for a quality improvement intervention that facilitated psychotherapy but more than $90,000 for a quality-improvement intervention that facilitated antidepressant use. For whites, the estimated cost per QALY gained was $30,000 for both interventions (14). A cost of $50,000 per QALY is generally considered cost-effective and a good use of health care resources (15).

Effectiveness trials also examine outcomes over a longer, more clinically relevant time frame. In their comparison of olanzapine versus haloperidol plus benztropine for schizophrenia, Rosenheck and colleagues (16) assessed traditional outcomes of psychotic symptoms and extrapyramidal side effects, as well as quality of life and medical costs for 12 months. Longer follow-up periods also allow for evaluation of longer-term side effects, including weight gain, hyperlipidemia, and insulin resistance (17). Partners in Care is currently examining five-year outcomes that include employment status and household wealth.

Because of their differences in methods, results from efficacy and effectiveness trials sometimes differ. For example, several randomized trials have suggested that olanzapine was superior to haloperidol in the treatment of schizophrenia in terms of symptom improvement, extrapyramidal side effects, and quality of life. However, Rosenheck and colleagues' study (16), which was conducted in 17 Department of Veterans Affairs medical centers with usual providers following a study algorithm, found no significant differences between patients assigned to haloperidol plus benztropine and patients assigned to olanzapine in terms of study retention; positive, negative, or total symptoms; extrapyramidal symptoms; or quality of life. Although olanzapine was less likely to produce akathesia and tardive dyskinesia, the drug also had greater potential for weight gain and increased cost. These findings may differ from those of previous efficacy trials at least in part because previous studies compared second-generation antipsychotics with haloperidol without prophylactic benztropine. Thus the comparator did not reflect "the next-best choice as used in optimal real-world practice" (18).