Comparing Psychotherapy and Pharmacotherapy

To the Editor: In his review article, Donald F. Klein, M.D. (1) critiqued the work of my colleagues and me in several ways. Dr. Klein asserted that “Gould et al. [2] performed only a MEDLINE database search that was limited to 20 years, from 1974 to March 1994” (pp. 1205–1206). This statement is incorrect. In fact, we also performed a CD-ROM PsycLIT search for the same years, examined secondary references to locate articles, and included studies that were in press at the time of publication that we had knowledge of because of their presentation at national conferences.

Dr. Klein argued that three of our 43 studies were “problematic” and should not have been included in the meta-analysis and that another study should have been included. We agree with Dr. Klein that the Black et al. study (3) should have been included and are unclear as to why our original MEDLINE search did not capture this study. We also agree with Dr. Klein that the subjects in the study by Charney et al. (4) were not properly randomized and should be excluded. Dr. Klein argued that the study by Klosko et al. (5) should not be included for several reasons, one of which was that “Since alprazolam is effective in the treatment of panic disorder, this trial lacked assay sensitivity (i.e., the ability to detect specific treatment effects)” (p. 1206). Was Dr. Klein saying that a pharmacotherapy study was invalid if it did not replicate previous pharmacotherapy findings? We agree with McNally’s comments (6) that Dr. Klein’s assertions (7) about assay sensitivity suffered from circular reasoning: when a manipulation check and an outcome measure are the same, there exists an inherent tautology. Independent criteria are needed to establish assay sensitivity.

It is of interest that even if we agreed wholeheartedly with Dr. Klein and had eliminated three studies from our meta-analysis and added one, the results and conclusions from our original work would have remained unchanged. Our original analysis yielded an effect size of 0.68 for cognitive behavior therapy and 0.47 for drugs in 43 studies. A reanalysis with these changes resulted in an effect size of 0.65 for cognitive behavior therapy and 0.49 for pharmacotherapy in 41 studies. The conclusions of our original analysis remain the same: pharmacological and cognitive behavior therapy both produce better results than contrast conditions, and cognitive behavior therapy is at least as effective as pharmacotherapy in the treatment of psychiatric disorders.

Finally, Dr. Klein (1) asserted that “Gould et al. [2]…included nine studies that lacked any contrast group, making it difficult to understand how a comparative effect size was calculated” (p. 1206). This statement is incorrect; all of our studies were required to have control conditions that included no treatment, wait list, drug placebo, or psychological placebo, and from these we derived effect sizes.