Integrating Predictive Modeling Into Mental Health Care: An Example in Suicide Prevention

A large, local VA health care system implemented an intervention for veterans identified as “high-risk for suicide” by a predictive model that analyzes data from medical records. The implementation described below is intended to represent a single example of how a predictive model was integrated into a hospital setting and does not represent the broader national VA implementation of the program.

The VA, in collaboration with the National Institute of Mental Health, developed the predictive model that identifies veterans at statistical risk for suicide (8). The statistical details of the model have been previously described (8, 9). Briefly, an initial development and validation study was conducted (8), followed by retesting and a comparison of the potential benefits of a variety of machine learning approaches (9). The final statistical model uses a penalized logistic regression model with 61 variables collected from electronic health records that include a variety of risk factors for suicide (e.g., demographic characteristics, mental health diagnoses, prior suicide attempts, medications) (9). In the validation study that examined risk over one year (8), the predictive model succeeded in identifying a group with suicide rates that were 30 to 60 times higher than the overall sample. Because only 31% of the veterans identified in the highest risk group had been flagged by clinicians as “high-risk for suicide” through the VA’s medical records flag system, the model identified additional veterans at risk (8). In addition, those at high risk for suicide were also at risk for a variety of adverse outcomes, including other external causes of mortality and all-cause mortality.

A national prevention program known as REACH VET (Recovery Engagement and Coordination for Health—Veterans Enhanced Treatment) was rolled out to local VA facilities to make use of the predictive model results. The predictive model produces a monthly list of veterans who are identified as being at high risk for suicide. Each VA facility was asked to name a local coordinator at program launch who would be responsible for engaging providers and making them aware of patients identified as “high risk,” helping providers understand and choose next steps, and answering staff and veteran questions about the program. A Web-based dashboard was developed to identify local REACH VET patients and to help track required suicide prevention steps. Once veterans are identified as high risk, guidance from the national REACH VET program includes a review of the treatment plan (e.g., frequency of mental health appointments, use of evidence-based treatments, re-evaluate pharmacotherapy) and evaluation of any treatment enhancements that might reduce risk (e.g., creating a safety plan that details coping strategies and social and professional supports available during a crisis, increased monitoring of stressful life events, peer support, telehealth, and sending periodic caring letters to at-risk patients). Providers are asked to reach out to veterans to offer support, assess risk, and collaboratively consider treatment changes.

At the time of writing, our health care system has had 313 veterans identified as high risk for suicide by the statistical model. Providers have considered additional prevention services for 100% of patients. In 66% of cases (N=207), the provider elected to reach out to the veteran as part of their review of treatment. Interpretation of this statistic is challenging, because many patients may have had upcoming appointments. Interestingly, only 7% of providers (N=22) documented that they informed the veteran that he or she was identified as being at high statistical risk for suicide or other adverse outcomes as part of that outreach. It is possible that this step was not specifically documented, or providers may be unsure how to communicate the complex meaning of statistical risk to veterans. Alternatively, providers may have believed that sharing this information was clinically contraindicated.

One of the beneficial outcomes of our local efforts was that the national REACH VET program subsequently modified and adopted the standardized note template described above for use across VA facilities nationwide.

Our facility learned important lessons during the implementation of REACH VET and established some best practices that were later adopted nationally. Rapid nationwide implementation of an innovative suicide prevention effort is a complex endeavor. One key to local implementation was the national REACH VET leadership and facilitator team who provided critical support to regional coordinators. The national REACH VET team was highly responsive to coordinators’ e-mails and calls, rapidly adapted the program based on feedback, and efficiently fixed technical problems. It is difficult to overstate the positive impact of the national team to our local implementation.

Several lessons from the implementation may apply broadly to mental health programs seeking to implement interventions that are based on predictive learning results. Without effective program leadership at the top, our local implementation would not have been as successful. Additional keys to success included local leader prioritization of the program, clear communication and preparation of the clinical staff before rollout, and dedicated and protected time for the coordinator. Local implementation often requires ongoing innovation, and our initial program data suggest areas for additional research, including whether and how providers should explicitly discuss suicide prediction results with patients. Additional research needs to include both clinical outcomes (e.g., effect of the intervention on suicide) and implementation outcomes (e.g., fidelity of intervention).

This column is the result of work supported with resources and the use of facilities at the VA Puget Sound Health Care System, located in Seattle and Tacoma, Washington. The contents do not represent the views of the U.S. Department of Veterans Affairs or the U.S. government.