Interprofessional Differences in Disposition Decisions: Results From a Standardized Web-Based Patient Assessment

The Massachusetts General Hospital department of psychiatry’s quality improvement team—consisting of the department’s director of quality management, director of postgraduate medical education, database manager, and recredentialing officer—designed a standardized Web-based simulation focusing on the implementation of best-practice guidelines for assessment of a patient’s risk of self-harm and violence. The standardized virtual patient was a male veteran who had returned six months earlier from Operation Enduring Freedom in Afghanistan. He expressed somewhat ill-defined suicidal ideation and plans, with questionable intent. He also expressed vague thoughts of violence toward his ex-wife.

The simulation was presented with the computer simulation assessment tool (CSAT) platform (9), which enabled participants to act as clinicians and perform a risk assessment for a standardized virtual patient during a Web-based, time-limited encounter. In order to do this, participants selected actions from drop-down menus and viewed the corresponding video clips; if a participant chose the “anxiety” action, for example, he or she was directed to a video clip of the standardized virtual patient describing his anxiety. [A full list of actions and the information revealed in each action are included in a supplementary table available online as a data supplement to this report.]

When the participants were finished with the simulation, either because the allotted 15 minutes had elapsed or because they felt they had sufficient information to make a decision, they were prompted with the statement, “Based on the information obtained during this evaluation, the appropriate level of care for this patient is:” and were then instructed to select a level of care from a list of options, which included outpatient follow-up in three months, two months, two weeks, or one week; partial hospitalization; inpatient hospitalization; and admittance to the emergency department. The simulation was intentionally designed so that there were many acceptable and unacceptable disposition decisions. Decisions to send the patient to an inpatient unit, emergency department, or intensive outpatient program were considered correct, and outpatient care with long-range follow-up was considered incorrect.

The simulation was disseminated via e-mail to all clinicians credentialed to treat patients within the psychiatry department of a large academic medical center as part of an annual recredentialing procedure between April and May 2012. Because participation in this ongoing professional practice evaluation assessment was a mandatory step in clinicians maintaining their appointment, a response rate of 100% was expected. Under an existing Partners HealthCare Institutional Review Board approval for educational activities, this activity was exempt from written informed consent requirements.

Binary logistic regression was used to test predictors of recommending emergency department evaluation and inpatient care (emergency department or locked inpatient unit, for example) versus outpatient care (weekly outpatient visits, monthly outpatient visits, or partial hospital program). In addition to professional background (physicians versus nonphysicians), we also examined the effects of gender, years since graduation, and practice setting within the department (ambulatory versus nonambulatory). In this study all of the physicians were psychiatrists. Finally, chi square tests were used to test whether providers differed in their choice of virtual actions taken during the CSAT, coded in a binary fashion as chosen or not chosen. If the difference was significant, we included these actions in the logistic model to test whether they were related to level-of-care recommendations.

Overall, 410 of the 412 clinicians in the department completed the simulation. Only the 358 participants for whom full demographic information was available were included in the analyses. The mean±SD age of participants was 51.6±11.5. There were more physicians (60%, N=216) than nonphysicians, and nonphysicians had diverse training backgrounds (76% [N=108 of 142] had a Ph.D.; 11% [N=16 of 142], an Ed.D.; 7% [N=10 of 142], a Psy.D.; 3% [N=4 of 142], an M.S.; and 3% [N=4 of 142], an M.A.). Genders were about evenly represented (53% female, N=190). Participants had been practicing on average for 21.47±12.06 years, but on average there was no difference in the number of years in practice for physicians versus nonphysicians. Practice settings (outpatient versus inpatient) did not differ significantly between the two training groups.

The binary logistic regression indicated that physicians were more likely than nonphysicians to recommend inpatient follow-up care (odds ratio [OR]=2.67) (Table 1). After other clinician-level factors were included in the model (model 1), training background continued to be a significant predictor of disposition choice (OR=2.86). Gender, years since graduation, and practice setting were not related to disposition decisions. Within the CSAT encounter, physicians differed from nonphysicians with regard to five actions. Specifically, physicians were more likely to assess psychosis (88% [N=191] versus 76% [N=108] of nonphysicians, χ²=9.52, df=1, p=.002), assess neurovegetative symptoms (82% [N=177] versus 73% [N=104], χ²=3.84, df=1, p=.05), discuss the patient’s legal history (75% [N=162] versus 64% [N=91], χ²=4.93, df=1, p=.03), and discuss the patient’s willingness to be hospitalized (81% [N=174] versus 66% [n=93], χ²=10.30, df=1, p=.001). Physicians were less likely to contract for safety than clinicians with other training backgrounds (61% [N=132] versus 87% [N=123], χ²=27.20, df=1, p<.001).

When these CSAT actions (model 2) were included in the logistic regression model, training background was no longer a statistically significant predictor of level of the clinician’s recommended follow-up care. Instead, contracting for safety and discussing willingness to be hospitalized emerged as significant predictors of level-of-care recommendations, where contracting for safety was related to a decreased likelihood of recommending inpatient treatment (OR=.10) and discussing willingness to be hospitalized was related to an increased likelihood of recommending inpatient treatment (OR=11.87). The other CSAT actions on which physicians and nonphysicians differed were not related to follow-up care recommendations. Taken together, these findings suggest that differences in follow-up care recommendations between physicians and nonphysicians were evident within the standardized patient encounter, as expressed by their choices to engage in either contracting for safety or discussing hospitalization. [The full analyses comparing physicians and nonphysicians on each action is included in the online data supplement.]

This study identified significant differences in disposition decisions based on training background, with physicians more likely to refer the standardized virtual patient to inpatient hospitalization or emergency department levels of care and nonphysicians more likely to recommend outpatient treatment. This effect was not explained by any other clinician-level factors, such as gender or years since graduation. The difference between physicians and nonphysicians in their treatment decisions emerged within the patient encounter, where, after accounting for diverging preferences for contracting for safety versus discussing hospitalization, we found that training background no longer predicted level-of-care recommendations. That is, compared with nonphysicians, physicians were more likely to discuss hospitalization, and this action predicted increased likelihood of inpatient follow-up care (almost 12 times as likely). Conversely, nonphysicians were more likely to contract for safety, which predicted a substantially reduced likelihood of inpatient treatment (approximately ten times less likely).

Although these two choices were associated with the differences between the decisions of physicians and nonphysicians, they were likely to be correlates of the decision and not causes for it. Both options were typically chosen within a few virtual actions of the final decision, suggesting that participants may have already made their disposition decision but wanted to evaluate the feelings of the standardized virtual patient to confirm their plan.

It is possible that the different training and experience backgrounds between physicians and nonphysicians led to different choices concerning disposition. For example, physicians spend a significant amount of time training in inpatient settings, which may make them more likely to choose inpatient hospitalization. In addition, previous research has found that psychiatrists sometimes make decisions about admission on the basis of concerns about liability (10). This may explain the decision not to contract for safety; although a contract can facilitate a therapeutic alliance, it does not reduce the clinician’s liability (11). Alternatively, it is also important to note that the participants had 32 different virtual actions to choose from, that participants differed significantly on only five of them, and that only two actions were related to disposition decision. This finding suggests that although training and experience for physicians and nonphysicians generally lead to similar actions, the few that differ can have a significant impact.

Despite the fact that this simulation was designed to be ambiguous and has no correct answer, the lack of concordance in disposition decision between professional groups working in the same hospital with the same patients is problematic, particularly with the recent emphasis on team-based care. In order to achieve patient-centered care, it is imperative that clinicians within a medical community share a common decision-making process. One study found that level of assessed risk had significantly higher predictive validity regarding later violence when raters (physicians and nurses) were in agreement than when they were not (12). However, this finding could be due to the complexity of cases, which would suggest that further training is specifically important in complex cases in order to come to consensus about patient care. The disparities across professions identified in this simulation will inform the design of future educational interventions for facilitating coordinated care. By elucidating differences between providers, this study highlights the potential need for different types of training for each provider type to bring all providers in line with standardized best practices of care.

Limitations of this study include external validity, in that it is unclear whether clinicians’ decisions for the standardized virtual patient reflected what they would do with a real patient. However, this structure was necessary to conduct a standardized comparison of a large number of clinicians’ decision-making processes, allowed for the identification of training and performance differences, and has been used in other studies to examine topics such as physician bias (13). Further, this sample was restricted to the psychiatry department of one academic medical center which may affect generalizability of the findings. However, this restriction ensured that all clinicians had a shared understanding of the relevant inpatient and emergency departments. In addition, the analysis did not account for other experience or training that providers may have had prior to their current position and practice setting.

This study indicates that, in an encounter with a virtual standardized high-risk patient, training backgrounds predicted disposition decisions among mental health clinicians, a clinician-level factor that may cause interprofessional disparities in patient care. The study also demonstrates that a Web-based standardized simulation is a scalable way to detect differences in how clinicians approach a complex patient.

Dr. Alpert has received honoraria from the Massachusetts General Hospital Psychiatry Academy and from Belvoir Publishing. The other authors report no competing interests.