Limitations of Evidence-Based Medicine Designs
Given the current lack of a model that relates goals and objectives to outcomes, it may be tempting to pursue an RCT, for example, to establish the clinical effectiveness and cost-effectiveness of IMPUs. With an RCT, it is not necessary to have a model that interlinks goals, objectives, and outcomes, because as long as the targeted outcome of an RCT is defined, the RCT will provide information on the variables that have been experimentally controlled. Thus, a well-executed RCT showing significant effects on long-term patient outcomes, such as survival or quality of life, would provide strong evidence for the success of the IMPU interventions or design choices tested in the trial. However, randomized designs run into several problems when deployed in IMPU studies, as detailed below.
The patients an IMPU serves represent a very heterogeneous group, and the IMPU intervention consists of a range of components that are not always stable over time, such as IMPU context and design, nurse training, physician team composition, and multidisciplinary collaboration. Moreover, it is difficult to randomize the routing of patients through a medical institution. In the case of IMPUs, randomization would involve three staff types: the physician or nurse who makes the call for psychiatric help, the IMPU staff, and the psychiatric consultation-liaison service, which in most cases would be the control condition. Patients and physicians are likely to have strong preferences for either the intervention or care-as-usual arm, probably influencing the randomization. In addition, when a patient is in a mental or behavioral crisis, researchers cannot directly obtain informed consent for study participation from the patient.
Although the problems with randomization could be addressed in theory, they will significantly bear on the design and outcomes of an IMPU study, and they may also interact. Furthermore, sampling bias may occur. For instance, three different staff members will have to consent to supporting a trial. At best, a subgroup of consenting patients could be randomly assigned to interventions of the trial. It is likely that some patients with more serious behavioral problems will not agree to study participation. Clinicians will probably want to include patients who do not have severe behavioral health problems and need to believe firmly that only empirical evidence based on an RCT justifies treatment allocation. Physicians have little incentive to have patients with disruptive behaviors randomly assigned to care as usual, because such patients can put strain on regular wards. It is likely that physicians would include less serious cases in the trial and would request direct access to the IMPU for patients with the most complex conditions.
It also will be challenging to convince caregivers that treatment in a department specifically equipped for the needs of those in their care may be withheld from them. Such an RCT would have a satisfactory internal validity, but it would not represent real-life patients and hospital staff. Moreover, the difference in the effect of the IMPU and care as usual will likely be limited, because it is less likely for patients in both conditions, owing to the abovementioned mechanism of sampling bias, to show disruptive behavior. Because disease complexity among these patients will be limited, this limitation will not allow IMPUs to use their full range of capabilities in treating patients with complex conditions and may not enable researchers conducting the trial to find a meaningful difference between intervention groups.
Necessary Conditions
In search of evidence for effectiveness of an IMPU, looking for circumstantial evidence may be helpful. Such an approach is in contrast to approaches that seek to identify a sufficient condition for evidence, such as RCTs that measure survival or quality of life. If an IMPU has convincing clinical and other effects compared with treatment as usual, the IMPU would represent a “sufficient condition for effectiveness.” For example, Kishi and Kathol (
16) suggested that a high-acuity IMPU is the most cost-effective IMPU type. Instead of conducting an RCT that compares the entire IMPU with care as usual, researchers may investigate whether the IMPU at a hospital meets the design described above and how the specific design element, that is, primarily serving patients with high acuity, is linked to costs. Researchers could then explore other necessary conditions, such as the link between joint patient rounds and satisfaction among patients, caregivers, and referring physicians. This approach does not investigate whether an entire IMPU is effective but whether design elements of an IMPU are associated with increased effectiveness.
Instead of verifying effectiveness of IMPUs as an integral unit, researchers may look only at specific conditions for the desired outcomes of an IMPU, without investigating whether these outcomes are interconnected. For example, staff satisfaction may be necessary for improving patient-related outcomes. Likewise, the key features or design elements introduced by Chan et al. (
13) might also be conditions that lead to several interconnected outcomes. The authors argue that IMPUs cannot function without buy-in from all levels of hospital staff; adequate communication between internists and psychiatrists; clinical support of social workers and physical, occupational, speech, and recreational therapists; and a robust triage system (
13). Each of these design elements could be investigated with respect to specific goals, such as decreased use of constant patient supervision and restraints, reduced LOS, and increased staff and patient satisfaction, to determine whether these elements are linked to specific outcomes.
Such proposed research lacks a parallel control group, and researchers can usually use only a pre-post design to evaluate a new treatment. However, if the goals, objectives, and other elements are formulated precisely, the need for a control condition is diminished. For instance, if IMPUs are open to nightly admissions and nightly use of patient restraints on the medical wards is reduced, IMPU care has helped reduce the need for restraining patients. This would be a solid, yet indirect, indicator for the effectiveness of the IMPU.