Comparative Effectiveness of Antipsychotic Drugs

To the Editor: This double-blind study enrolled 157 patients with chronic schizophrenia and schizoaffective disorder and compared the efficacy and the safety of three atypical antipsychotics (clozapine, olanzapine, and risperidone) with one another and with haloperidol. The analysis of the results was based on two statistical approaches: 1) analysis of covariance for determining change over time in symptom scores, with baseline severity as a covariate, and 2) random regression with hierarchical linear modeling. As the authors stated in their section on Statistical Analyses, after a preliminary analysis of the results, “Hierarchical linear modeling analysis was adopted as the primary statistic for our study” (p. 257) and was in fact used to test the significance of differences in symptom severity.

In our view, this approach is incorrect. The traditional method of conducting clinical trials requires the investigators to predetermine both the endpoints and the analyses and to carry out the statistical analyses originally planned regardless of what happens with random assignment of subjects. So the sequence in design and performance of the trial that we expected was the following: 1) performance and completion of the study and 2) execution of the analyses initially planned by the protocol (and possibly a third phase in which other analyses were carried out after the realization that random assignment to groups was unsuccessful in some respects).

On the contrary, the sequence of work by Dr. Volavka et al. was the following: 1) performance and completion of the study, 2) preliminary analysis to determined what happened with randomly assigned groups, and 3) determination of the most convenient analysis in light of the results provided by the preliminary analysis. Of course, phase 3 can be criticized because it is clearly biased.

We appreciate that the article was very honest on this point because it stated that the analysis was chosen after observation of the results. However, if the analysis is conducted with this open method, the double-blind design makes little sense, and more important, the results of the study become less reliable.