STAR*D Level IV Methodology

To the Editor: The article by Patrick J. McGrath, M.D., et al. (1) based on STAR*D trials was an important randomized trial of refractory depression. We would like to address two methodological issues in the analysis, however, which may impact its clinical implications.

First, according to Table 1, 41.4% of the patients randomized to the tranylcypromine group entered this study because of previous medication intolerance with other STAR*D trials, while only 21.6% of the patients randomized to the venlafaxine/mirtazapine group had previous medication intolerance. This imbalance, which was not adjusted in the statistical analysis, might have biased the results against tranylcypromine in the tolerability measure, casting doubts on the finding that participants taking tranylcypromine were more likely to exit the study because of side effects.

Second, the primary outcome of remission, based on the Hamilton Depression Rating Scale, was similar in both groups and thus a negative result. However, the secondary outcome of treatment response, based on the Quick Inventory of Depressive Symptomatology-Clinician-Rated, was 12.1% for tranylcypromine versus 23.5% for venlafaxine/mirtazapine. The article downplayed this difference by stating that “response rates were also low and not significantly different” ( 1, p. 1535). Hypothesis testing methods are most appropriate when the study is designed and powered to test the hypothesis; otherwise, the most appropriate statistics for secondary outcomes and other post hoc analyses are effect estimates and confidence intervals (2, 3) . The p value for assessing significance in this comparison is less relevant because the study was powered to assess the primary outcome measure of remission rate, not response rate (3) . Hence, type II error is important: when assessing outcomes that the study was not powered to assess, one cannot equate statistical nonsignificance with mathematical nondifference or clinical nonsignificance (4) . We recalculated the relative risk between the two groups, which was 1.95, with a 95% confidence interval of 0.83–4.57. Although the null is included, these results are most consistent with a probability of an effect. The most likely clinical conclusion, pending further studies, could thus be that the venlafaxine/mirtazapine combination may have moderate benefits over tranylcypromine based on response rate.