Dr. Bauer Replies

To the Editor: Dr. Mattes’s comments are extremely important. He correctly points out that evidence-based medicine analyses are limited by the availability of relevant data—typically published studies. Even the Cochrane Collaboration, the premier evidence-based medicine review group (1), is limited primarily to published or soon-to-be published data. Should we worry that the peer-reviewed, published data on which we based our evidence-based decisions are biased? The answer is overwhelmingly, “Yes, worry!” For instance, meta-analyses of antidepressant and anxiolytic efficacy, based on predominantly published data, are regularly positive (1). However, analysis of the Food and Drug Administration’s (FDA’s) Summary Basis of Approval Reports for approved antidepressants and anxiolytics—which reflect all studies, published and unpublished, submitted to the FDA—indicates that the majority of such studies are actually negative (2). Concern is not limited to clinical trials but extends to pharmacoeconomic analyses as well: industry-sponsored studies are significantly more likely than non-industry-sponsored studies to favor their proprietary drugs over comparators (3). Potential bias in the published, peer-reviewed evidence base is not limited to the mental health literature. This issue has in fact been investigated much more extensively in medicine (e.g., references 3–6). Of additional concern, it has been alleged that the pharmaceutical corporations have frequently been in violation of a 1997 law mandating reporting to the FDA of all clinical trials of a treatment when seeking FDA approval (7), thus compromising even the unpublished data set in the public domain. The impact of such practices on psychiatric research, practice, and public health has simply yet to be delineated. Regardless of how even-handed evidence-based reviews of the peer-reviewed literature are, they are still limited by what is actually published. Dr. Mattes’s message is appropriate: Caveat lector!