The STAR*D Data Remain Strong: Reply to Pigott et al.

TO THE EDITOR: We wish to respond to a reanalysis of our original STAR*D data (1) that recently appeared in BMJ Open (2). This re-analysis concluded that the cumulative remission rate of depressed outpatients undergoing four to five sequential antidepressant therapies across approximately 12 months in the study was 35%, instead of the 67% rate reported by Rush et al. (1). The Rush et al. paper was prepared in response to an invitation from the Editor-in-Chief of the American Journal of Psychiatry at that time to submit a paper that would provide a comprehensive overview of the findings of the entire STAR*D trial, since all the primary outcome papers had been already published in major journals, including the American Journal of Psychiatry.

The analytic approach taken by Pigott et al. (2) has significant methodological flaws. Pigott et al. selectively eliminated the data from 561 (15%) of the 3,671 patients reported by Rush et al. (1) who enrolled into Level 1 of STAR*D, 297 (21%) of the 1,439 patients reported by Rush et al. (1) who enrolled into Level 2, 80 (21%) of the 377 patients reported by Rush et al. (1) who enrolled into Level 3, and 3 (3%) out of 109 patients reported by Rush et al. (1) who enrolled into Level 4. In total, 941 patients included in our original analyses were eliminated from Pigott et al.’s reanalyses based on their post-hoc criteria. The rationale for removing these participants from the longitudinal analysis appears to reflect a studious misunderstanding of the aims of the Rush et al. paper, with the resulting large difference in remission rates most likely the result of exclusion by Pigott et al. of hundreds of patients with low symptom scores at the time of study exit.

The overall goal of STAR*D was to conduct a series of randomized comparisons of the effectiveness of a number of commonly used antidepressant medications and adjunctive strategies across three steps (Levels 2, 3, and 4) in a representative sample of depressed outpatients. To enter the sequential comparative effectiveness trials, patients first were treated for up to 3 months with the antidepressant citalopram (Level 1). Effectiveness trials by design aim to be more inclusive and more representative of the real world than efficacy trials. By removing the data of over 900 study participants from their reanalyses, Pigott et al. failed to recognize the purpose of inclusiveness. It appears that the authors created rules to define post hoc which subjects to include, which eliminated many subjects who experienced large improvements during one or another of the study’s levels. By doing so, the sample is biased to underestimate the actual remission rates.

Our original report of the Level 1 outcome in STAR*D (3) was in fact criticized by a number of researchers for underestimating remission rates in Level 1, as Pigott et al. (2023) mention: “STAR*D investigators state in their level 1 article, ‘our primary analyses classified patients with missing exit HRSD scores as nonremitters a priori’ (Trivedi, 2006).” This approach, which also was used in the primary analyses of each of the randomized treatment levels, is very conservative because some patients who drop out of studies are actually improved at the time of exit from the study. One of the limitations of STAR*D was the fact that the Hamilton Rating Scale for Depression (HRSD) was only administered at baseline and at the end of each level (typically after 3 months of treatment). As a result, HRSD scores were typically not available when patients dropped out of the study. By contrast, the patient-reported Quick Inventory of Depressive Symptomatology-Self-Report (QIDS-SR), a well-validated measure of depression severity (4), was administered at every patient visit and was therefore deemed by the authors to provide a more accurate reflection of the patients’ clinical status during the trial. It is exactly for this reason that the QIDS-SR was used in the Rush et al. (1) paper, since the QIDS-SR captured the patients’ symptom status regardless of level/step and regardless of whether the HRSD was obtained at study exit.

A primary criticism leveled in the Pigott et al. paper is that “the STAR*D investigators did not use the protocol-stipulated HRSD to report cumulative remission and response rates in their summary.” What Pigott and colleagues fail to appreciate is that the overall outcomes of patients across 1 year of treatment reported by Rush et al. (1) was not an “a priori”-identified analysis in the protocol (5) but a secondary “post-hoc” report, specifically requested by the Editor-in-Chief of the American Journal of Psychiatry at that time, with the goal of summarizing the clinical outcomes—as measured by the self-reported QIDS-SR (capturing the symptom status of each patient at the last visit regardless of level and regardless of whether or not the HRSD was obtained at study exit)—of this complicated multilevel trial. As such, the use of different methods and alternate measures in secondary analyses is a well-accepted scientific approach to explore the data and develop new hypotheses for future research. Moreover, as clearly stated in the Rush et al. paper, the estimated cumulative remission rate was based on the assumption that the patient remained in the study, completed it, and, if needed, participated in all four levels of treatment.

The large discrepancy in remission rates reported in two papers working with the same set of patient data is surely provocative but indicates that one of the conclusions is not plausible. Pigott et al. concluded that only 35% of depressed subjects achieved remission with up to four antidepressant treatments in the course of approximately 12 months. In Level 1 of STAR*D, remission rates were 28% based on the HRSD and 33% based on the QIDS-SR (3). Therefore, the finding of Pigott et al. is that only an additional 7% of the depressed patients achieved remission in Levels 2, 3, or 4. Our primary papers reporting the outcomes of those levels disprove that. We note that the senior author of the Pigott et al. paper, Dr. Jay Amsterdam, co-authored a paper reporting the results of a study in which he played a key role and that utilized a sequential pharmacotherapy protocol informed by the STAR*D results (6). They found a 60% cumulative remission rate across 12 months with antidepressant treatment alone, a result that is much closer to the 67% remission rate of the original Rush et al. STAR*D report than the Pigott et al. rate of 35%.