Dropouts and Missing Data in Psychiatric Clinical Trials

To The Editor: In the June 2009 issue of the Journal, the editorial by Robert M. Hamer, Ph.D., and Pippa M. Simpson, Ph.D. (1), discussed last observation carried forward and mixed models as described in the article by Carla M. Canuso, M.D., et al. (2) . Drs. Hamer and Simpson concluded that “mixed models are preferable to last observation carried forward when the number of subjects is sufficiently large and the proportion of missing data is small enough” (dropout rate <20%) ( 1 . p. 641).

However, since most psychiatric clinical trials report dropout rates of >20%, the problem of missing data remains a major challenge for analysis. The key issue is whether those who drop out differ from those who remain in the study and how this relates to individuals’ treatment outcomes. A simple graphical method we"ve previously described (3) addresses this matter directly. In a 40-week randomized study of ziprasidone and haloperidol, we divided the entire study sample into distinct subgroups corresponding to individual dropout times and included the completer subgroup. Missing mechanism assumptions could then be checked based on the longitudinal responses among completers and in each of the distinct dropout cohorts. Plotting the trajectory of Positive and Negative Syndrome Scale (PANSS) scores for the dropout cohorts showed that the missing completely at random assumption was violated and pointed to a key source of bias: the subjects receiving ziprasidone, 80–160 mg/day (standardized as a z score), who discontinued early in the first 6 weeks had higher PANSS scores than the haloperidol subjects, but this difference was not seen at other time points.

In the Canuso et al. study, the primary efficacy analysis was based on the last observation carried forward, which assumed that the responses following dropout remained constant at the last observed value prior to dropout, a not very likely event. A mixed-model repeated-measures analysis was also performed for observed cases occurring within a visit window at each time point. It is not clear whether PANSS scores obtained at all visits (and most importantly the dropout visit) were included in this mixed-model repeated-measures analysis (e.g., 24 PANSS scores missing at day 5, while only 20 dropouts during the 2-week monotherapy phase for the paliperidone group). The data supplement accompanying the online version of this letter summarizes the missing data pattern extracted from Figure 2 of the Canuso et al. article. Because no data are available to verify whether missing data at day-5 visits are the same as the set of observed values (missing completely at random assumption and ignorable) or whether they are dropout-related (nonignorable), it would be difficult to interpret the early responses to paliperidone therapy at day 5 relative to quetiapine (p<0.05) and placebo (p<0.001) (2) . The use of a dropout cohort plot would allow accounting for all dropouts at each study time point and the identifying of potential bias.