What Works for Whom? A Critical Review of Psychotherapy Research

Roth and Fonagy and their associates have provided us with an ambitious volume, reviewing and analyzing more than a thousand studies of psychotherapeutic outcome. The opening three chapters, which discuss in depth the issues involved in such an enterprise, are thoughtfully wrought and will undoubtedly achieve the status of a classic contribution in this area. The next 10 chapters are devoted to reviewing the incidence, natural history, and research findings on particular DSM-IV diagnostic categories (depression, anxiety disorders, etc.), followed by several miscellaneous chapters (on child and geriatric populations, for example), and a final chapter on conclusions and implications.

The authors are absolutely meticulous and unsparing in cataloging the formidable difficulties and limitations of outcome research. “Research into psychotherapy necessarily and inevitably changes the nature of the therapy it investigates,” they say (p. 13), a sort of Heisenberg Principle run rampant. They address further complicating issues, such as comorbidity, the general versus the specific impact of a modality, and the built-in bias of certain scoring instruments to influence, or even dominate, the apparent outcome (for example, the Beck Depression Inventory tilts toward cognitive therapies, and the Hamilton Depression Rating Scale may favor medication outcomes). In addition, the tortuous natural history of many psychiatric disorders means the timing of follow-up may all but determine a study's efficacy findings. The editorial decisions, to systematize by diagnostic group rather than by treatment modality, and to omit most open trials, introduce consequences of their own. The authors make abundantly clear that the lot of the follow-up researcher is certainly not an easy one.

Since the book grew out of a report commissioned by the U.K. National Health Service, the needs of “stakeholders” (in the United States, read “third-party payers”) receive prominent attention. At the same time, the authors are clearly committed to excellence in clinical practice and to encouraging productive interaction between researchers and clinicians. They encourage researchers to be aware of disparities between quantifiable efficacy under research conditions (homogeneous, randomly selected patient populations, closely monitored therapy, etc.) and effectiveness in the actual clinical situation (typically pragmatic, seldom applying an orientation in pure form, responsive to the clinical situation, etc.). They also encourage clinicians to attend to research findings and use them to base their clinical practice on evidence rather than on accidents of predilection, training, or history. “The challenge,” they say, “is to achieve practice that is rooted in empirical findings but not circumscribed by them” (p. 358).

In the authors' accounting, the highly researched cognitive behavior therapy and interpersonal therapy more often rate as “clearly effective” for their relevant patient groups, while psychodynamic therapies more often rate as “promising or having limited support for efficacy” for theirs. Although the relative paucity of research in psychoanalytic and psychodynamic therapies certainly warrants criticism, it is exaggerated here by the authors' decision to omit most of the research that does exist. The work of the Anna Freud Centre is liberally discussed in the excellent chapter on child disorders, but Wallerstein's longitudinal study gets only passing mention (of a brief overview paper [1] and not the definitive book [2]), and numerous existing studies receive no mention whatsoever. For one striking example among many, the series of papers from the Psychoanalytic Clinic at Columbia University (reference 3, for example) on 1,585 cases divided between psychoanalytic and psychotherapeutic treatment is nowhere to be found. These missing studies have presumably fallen victim to the authors' arguable conclusion that randomized controlled trials “provide the only valid—albeit limited—source of evidence for [therapeutic] efficacy” (p. 19). Although one hopes that psychodynamic therapists and psychoanalysts will heed the call to find more ways to systematize their outcome studies, one wonders why the authors confine their approval to the randomized controlled trial methodology when their own opening chapters have itemized the many factors raising questions about its suitability to psychodynamic therapies.

The reader comes away impressed by the magnitude of the project and by the complexities involved in studying efficacy in ways that will illuminate clinical effectiveness. In addition, one comes away with concern that, despite their many eloquent warnings to the contrary, the book's title and its very structure, with neat summaries at the end of each chapter rating therapies for efficacy, invite premature closure. “Basing payment decisions on research evidence of efficacy.needs to be undertaken with care if it is not to do more harm than good,” say the authors (p. 43), and they cogently point out the severe limitations that would have attended acceptance of the standard practices of the 1960s as controlling guides for grants, research, and practice. At the same time, however, as they issue these warnings, they issue an explicit invitation to “stakeholders” to consider the book's findings in their planning, which is perhaps a more starkly hazardous invitation to proffer to our managed care organizations than to the British National Health planners they may have had in mind.

We and the authors hope that the chapters on current outcome will be in a constant state of refinement and revision as ongoing research results come in. The illumination the authors provide to the terrain, however, and to the clinical importance of outcome research, will remain “must” reading for researchers who wish to communicate effectively with clinicians, as well as for clinicians open to incorporating research data into their clinical decision-making. In a sense, the pithy opening chapters outline the major tasks confronting us and tell us with just how many grains of salt we should approach the rest of the book. Despite the bold ring of their title, I believe the authors of this important effort would agree with the admonition with which Smith et al. ended their pioneer meta-analytic study of psychotherapy outcome: “Science issues only interim reports” (4, p. 189).