Efficacy vs. Effectiveness in Psychiatric Research

In Reply:Drs. Summerfelt and Meltzer raise thoughtful arguments that nevertheless can be questioned.

First, attributing "external validity" to effectiveness studies but only "internal validity" to efficacy studies seems inaccurate. The external validity of any treatment is based on legitimately inferred effects that have been demonstrated in a well-defined population, an important characteristic of efficacy studies.

Second, the real "major difference" between efficacy and effectiveness research is the profound inability of effectiveness studies to draw causal treatment relationships from the uncontrolled or quasiexperimental approaches that typically confound treatment and patient selection factors. The needs of patients who have been systematically excluded from efficacy studies can and should be addressed by efficacy studies relevant to specific patient populations.

Third, while a single efficacy study might indeed have limited generalization, validation in similar (but rarely identical) samples through the use of randomized clinical trials has traditionally Increased generalization. Generalizability is constrained by an absence of replication that also applies to the effectiveness study.

Fourth, "well-planned studies [that] use random assignment procedures within naturalistic settings to produce unbiased estimates of the benefits of interventions" are studies of efficacy rather than effectiveness. If not, then the distinction is made even more obscure. Control of bias is the methodologic cornerstone of efficacy studies. That effectiveness-research settings are extolled as being "naturalistic" is an oxymoron; they typically lack the resources needed to apply and test a psychiatric treatment competently. Once the requisite resources are in place, one has an efficacy study, no matter what the setting.

Effectiveness research in psychiatry remains an empty promise. I am not aware of any effectiveness study, with or without policy change, that has been conducted on the most efficacious psychosocial treatments for schizophrenic, affective, or personality disorders. Experience to date suggests that the debate over efficacy versus effectiveness has primarily served to justify therapeutic neglect and to siphon millions of dollars from controlled clinical trials.