Randomized Controlled Trials

To the Editor: We read with great interest the article by Murray B. Stein, M.D., and his colleagues (1), which reported on a double-blind randomized study. Randomized controlled trials are always cited as the gold standard for detecting treatment efficacy. However, they often can be flawed in design and are not immune to bias. The relevance of such a study has been criticized on the grounds of small group size, selection bias, and improper random assignment to groups.

The authors failed to provide information about how many subjects were initially assessed, how many subjects were excluded from study, and any reasons for exclusion. We do know the rate of participation, the rate of response, and the implications for generalizability and future research. The selection of patients in randomized controlled trials has been a controversial issue. In this context, the guidelines for the Consolidated Standards of Reporting Trials (CONSORT) state that all patients assessed for trials should be accounted for and that the report should be accompanied by a diagram that explains what happened to all of the patients involved in the trial (2).

Random allocation to intervention groups remains the only method of ensuring that groups being compared are on an equivalent footing at the outset of the study, thus eliminating allocation and confounding biases. In this particular article, information was lacking on both components of randomization, concealment and method. Improper randomization can introduce serious allocation bias. Assessment of the quality of randomization in published trials has consistently found flaws (3, 4). CONSORT guidelines have emphasized that methods of randomization should be clearly reported (2). It has also been reported that trials with adequate or unclear allocation concealment may yield larger estimates of effect than those with inadequate concealment. This exaggerated estimate of treatment effects reveals a meaningful level of bias. In the absence of adequate information on the methods of randomization, the possibility of allocation bias may be raised. Elimination of bias would have been possible if the authors of this article had strictly followed CONSORT guidelines.

Finally, the authors failed to communicate effect size precisely. Since they presented their results almost solely as p values, it is difficult to understand their meaning. It is possible for small effect sizes to become statistically significant, which is evident in this study. However, it is the precision of effect size rather than the level of significance that determines how much faith a critical reader will have in the authors’ results. We also do not know the response rate and the remission rate, which determine the clinical importance of the results.