Why Do Promising Medications Keep Failing in Phase 3 Trials?

In phase 2 drug trials, the agent is administered to patients with a specific disease or condition. These studies usually include hundreds of patients and are essential for establishing the effect size, dose range, general safety, and tolerability profile. Phase 3 trials are much larger and can involve up to 3,000 participants. These studies can confirm the phase 2 findings and provide additional safety information.

Increasingly, phase 3 trials have been global and have included as many as 200 different research sites. While several recently completed phase 3 trials suggest that the active drug resulted in similar symptom reduction as in the phase 2 trials, the placebo response was substantially greater, and the active drug did not separate from placebo.

At the 2017 meeting of the International Society for CNS Clinical Trials and Methodology, study sponsors used their databases to identify factors in trial implementation that may have led to negative results. In one large trial that focused on negative symptoms in schizophrenia, a number of factors were identified that appeared to be associated with a lack of separation between drug and placebo. These included entering subjects on multiple antipsychotics, including subjects who had inadequate drug exposure based on plasma levels, and including individuals who had substantial changes in negative symptoms between screening and baseline. A trial from another company also found that dropping subjects who had substantial changes in symptoms prior to randomization decreased the placebo response.

Among study sponsors, there was agreement that changes late in trials to increase the number of subjects tended to lead to an increase in the placebo response. These issues in implementation may occur because the financial incentives for investigators and contract research organizations, and even perhaps within the sponsoring companies, encourage enrollment with ambitious timelines. This may reinforce a tendency to focus on quantity rather than quality and to relax enforcement of entry criteria later in studies.

Assuming that phase 2 preparation has been adequate, there are four areas that we recommend researchers and companies look for remedies for trying to improve the conduct and effectiveness of phase 3 trials: patient selection, severity rating instruments and their use, study design and controlling for the effects of being in a trial, and sponsor policies and procedures regarding trial conduct.

Patient selection. Outside adjudication of patients selected by sites to ensure that they meet inclusion and exclusion criteria has been proposed as a way of independently addressing this potential problem (1). Some studies have tried blinding site investigators to threshold severity criteria to avoid score inflation. Others have set up registries to identify and exclude “fraudulent” patients (individuals who are not real patients who try to gain entry purely for financial gain).

Rating instruments. There is a tendency to stay with instruments that are familiar, but these are not necessarily always the best. Studies that are using endpoints such as negative symptoms or cognition batteries should explore methods for improving measurement. Use of more objective, biologically based measures should be explored. Patient self-report needs to be explored as perhaps a better approach for severity rating in at least some patient populations. In addition, for instruments that do require application by raters, how this is done is critical, and concerns about practices by site investigators have led to technologies to try to improve the quality of severity ratings, including video ratings or centralized raters.

Study design. There has long been concern that the decades-long practice of single-blind placebo run-ins as a way of identifying and excluding placebo responders is ineffective. One approach to this problem has been to have a double-blind placebo run-in in what is often referred to as a delayed-start design; however, this appears not to have been a successful design at reducing placebo response. A more nuanced version of the double-blind placebo run-in is the sequential parallel comparison design, and this has had some success in reducing placebo response (2).

Another observation that is commonly seen in failed studies is a dramatic improvement in symptoms across all treatment arms once patients are randomized, suggesting that being in a study itself may have a powerful therapeutic effect. It may be useful to try to simplify studies as much as possible, partly as a way of reducing interactions with study staff that may be one explanation for this improvement. Making sure that all study procedures, staff interactions, and assessments are included in any run-in period prior to actual randomization might help to reduce this effect.

Sponsor procedures and policies. Finally, one of the problems contributing to study failures might be company policy and commitment to study success. As a program moves to phase 3, studies usually involve many more sites, and the quality of sites may diminish. Such factors themselves may contribute to failure (3).

Sponsor policy should focus more on quality of recruitment and site conduct than on volume of patients enrolled, which may result in fewer but more productive sites. Such policy would mean ongoing monitoring of site and contract research organization conduct and dropping of sites that do not perform. It might also involve more careful monitoring of patients to make sure they are actually taking their assigned treatment (4). Providing incentives for quality performance, rather than the number of subjects and study visits, can be included in contracts. The study coordinator should have a personal commitment to seeing the study through and ensuring its successful conduct.

We recognize that these suggestions are currently untested. It will be important going forward to systematically collect data on the “efficacy” of these suggested approaches so that companies have some basis for deciding whether to add a particular approach to their programs. Systematically looking at the efficacy of a particular approach could be incorporated into a typical registration trial by randomizing addition of the approach to sites within a study and by testing the value of the addition on signal detection as an exploratory hypothesis, without compromising the overall goal of the study.

If systematic problems with methods for phase 3 trials are to blame for the failure of promising medications in phase 3 trials, it is vital for our field to address this problem because medications that can improve the lives of people with serious mental illness may never reach the patient population. Of great importance for our field, the failure of these trials appears to have convinced some companies to limit investment in the CNS (central nervous system) area and in psychiatry in particular, substantially decreasing the resources available for drug development. ■