Different Standards When Assessing the Evidence for Psychodynamic Therapy? Response to Cristea et al.

We decided to use “target symptoms” as the primary outcome because it is a disorder-specific and useful measure assessing change in the main problem area a patient presents with (e.g., depressive symptoms in major depression, weight gain in anorexia nervosa, suicidality in borderline personality disorder). This taps the symptoms most relevant to the disorder. By using “target symptoms,” a strict test for psychodynamic therapy is implied because other therapies such as cognitive-behavioral therapy (CBT) focus explicitly on target symptoms. In addition, we assessed “general psychopathology” and “psychosocial functioning” as secondary outcomes, with all analyses reaching the same conclusion. In fact, combining all outcome measures assessed, as done, for example, by Wampold and colleagues (2), reaches an effect where the value of g is −0.12 and the equivalence confidence interval is −0.20 to −0.05, thus again confirming our original finding. In addition, the type of diagnosis was not found to be a significant moderator of outcome, suggesting no differences across disorders.

Lumping together different forms of comparison treatments is a well-established approach in meta-analysis. For example, testing against “treatment as usual” can consist of vastly different types of treatments. Cristea and colleagues themselves regularly use such an approach, for example, in their recent meta-analysis on borderline personality disorders: “Given the diversity and complexity of therapy orientations, we used an inclusive approach in delineating the psychotherapy and control conditions.… No constraints were placed on the control group, which could include (but was not restricted to) treatment as usual or other treatments not specifically developed for [borderline personality disorder]” (3, p. 320). In contrast, we included only comparison treatments with established efficacy, making this a much more homogeneous comparator despite variations in the CBT conditions. Between-study heterogeneity also was very low.

For their critique on equivalence testing, Cristea et al. cite an article by Treadwell and colleagues (4). However, Cristea and colleagues seem to have misunderstood what this article is about (i.e., evaluating individual trials self-identifying themselves as equivalence trials). This is a conceptual difference that cannot be directly transferred to our meta-analysis. While we agree that defining an equivalence margin is challenging, we do not see why equivalence trials or meta-analyses are particularly prone to bias. The same is true for our preference of intent-to-treat data. Both intent-to-treat and completer data are not optimal, and a researcher has to prespecify which kind of data is to be included in the analysis, which we did in our protocol. It is open to further research whether intent-to-treat analyses carry the risk of diluting treatment differences (5, 6). In our meta-analysis, only 10 (out of 23) randomized controlled trials provided intent-to-treat data, and in these cases the primary outcome was reported only for the intent-to-treat population. Thus, we used the data that were reported.

It is true that our meta-analysis was funded by a professional psychoanalytic society. The sponsor was not involved in conducting this meta-analysis. In addition, we controlled for allegiance on both the level of performing this meta-analysis (by including two cognitive-behavioral colleagues, one of whom holds the chair of behavioral psychotherapy at TU Dresden) and on the study level by using the multilevel allegiance rating scale.

It is true that equivalence trials make sense only if control interventions proved efficacious for the condition studied. That is exactly why we ensured the efficacy of the comparator. Cristea and colleagues seem to assume that this needs to be the case within the trials included in the meta-analysis. However, the efficacy of the comparison condition needs to be established in principle, not necessarily in the trials being included. In both the study by Zipfel et al. on anorexia nervosa (9) and the study by Crits-Christoph et al. on cocaine dependence (10), CBT was not superior to comparison conditions, one being an enhanced version of treatment as usual (9), the other being an established treatment (i.e., individual drug counseling based on the 12-step program [10]). However, CBT is considered established (11) for these conditions, independent of the outcome of these two trials, and thus was included in the meta-analysis. Therefore, the key assumption of assay sensitivity was not violated (4).

It is possible that results of an individual study differ from those of a meta-analysis. However, we agree that results of psychodynamic therapy in bulimia are controversial (12–14) and that further research on bona fide psychodynamic therapy in bulimia is required.