Connecting Research and Practice: Implementation of Suicide Prevention Strategies in Learning Health Care Systems

We previously described an evolving model of a learning health care system in the Mental Health Research Network (MHRN) (mhresearchnetwork.org) (1), a network of 14 research institutes embedded in health care organizations across the United States. National Institute of Mental Health (NIMH) funding supports infrastructure work, including establishment and maintenance of a virtual data warehouse at each site as well as pilot and signature projects.

MHRN-affiliated health care systems care for and insure 25 million patients across 16 states, with embedded mental health researchers who conduct federally funded research and maintain relationships with health care system leaders and clinicians at each site. This engagement varies and includes researchers attending leader and departmental meetings to learn of system priorities and disseminate research findings, engaging with or working as frontline clinicians, and serving on or acting as advisors to health care system committees. Challenges to building and maintaining these partnerships include competing organizational priorities, limited leadership and clinician bandwidth, and the perception of research as being too slow to meet the demands of pressing clinical decisions (with leaders often required to make decisions in weeks, not years). Researchers have addressed these challenges by scheduling quarterly meetings with leaders, supporting organizational priorities, identifying and leveraging overlap between the funding agency and organizational priorities, serving as interpreters of external evidence and generators of internal evidence, and designing pragmatic trials that can quickly adapt as needed.

MHRN health care systems were early adopters of the PHQ-9 to screen for and monitor depression and suicide risk. Responses to item 9 in the PHQ-9 data revealed that approximately 6% of respondents reported thoughts of suicide more than half the days in the previous 2 weeks, with 0.5% attempting suicide in the next 30 days and 3% within 2 years (2). These findings led to four streams of clinical or research activities in our health care systems: use of the Columbia Suicide Severity Risk Scale (CSSRS), implementation of the Suicide Prevention Trial (SPOT), evaluation of Zero Suicide implementation, and use of machine learning to improve suicide risk prediction.

Despite evidence that elevated scores on item 9 of the PHQ-9 were reasonably good at identifying people at increased risk of suicide, there is no evidence that suicide screening by itself prevents suicide attempts or deaths. Despite this evidence gap, health system leaders felt compelled to act and implemented workflows for patients reporting suicidal ideation. In many systems, these workflows led to systematic use of the CSSRS for patients with elevated scores on PHQ-9 item 9. Implementation varies across health care systems, but most prompt completion of the CSSRS in the electronic health record (EHR), followed by actions such as further clinical assessment of risk or protective factors, lethal means counseling, and/or completion of EHR-based suicide safety plans. The clinical use of the CSSRS in MHRN health care systems provides an opportunity to study whether clinicians use the CSSRS, whether its use prompts subsequent clinical actions, and whether these actions influence suicide attempts.

Researchers and health system leaders also sought to understand the limitations of self-reported measures, such as the PHQ-9 and CSSRS, and how to improve these measures. In one health system, researchers conducted semistructured qualitative interviews with suicide attempt survivors (3) and primary care patients (4), who described the value of being asked about suicidality but also how disclosing suicidal thoughts often involved weighing hope for help against fears of negative consequences associated with stigma and loss of autonomy. This research confirmed and extended research among veterans who reported similar fears and underscored the importance of direct and caring communication about suicidality and relationships with trusted providers (5). Recognizing these limitations, MHRN investigators are collaborating with health system leaders to improve screening and assessment practices and are developing and evaluating alternative methods to identify risk using medical records data, discussed below.

To generate evidence to guide large-scale secondary interventions for populations at risk of suicide, we conducted a pragmatic trial enrolling 18,882 patients (67.3% [N=12,701] women, 0.8% [N=158] American Indian or Alaskan Native, 3.1% [N=579] Asian, 0.4% [N=82] Native Hawaiian/Pacific Islander, 4.2% [N=797] Black or African American, 77.1% [N=14,558] White, 8.3% [N=1,563] Hispanic, 3.4% [N=640] more than one race, 11.0% [N=2,068] unknown or not reported) from March 2015 through September 2018 across four health care systems (6). Patients who reported suicidal ideation on the PHQ-9 were randomly assigned to receive ongoing usual care or one of two interventions: an online dialectical behavioral therapy skills training program supported by a health coach or a phone-based care management intervention designed to keep patients connected with their behavioral health clinicians. Of note, SPOT care managers used the CSSRS to assess suicide risk and to inform risk-based care pathways, which will ultimately contribute information about whether the CSSRS may be useful for suicide prediction and prevention. The primary outcome was suicide attempt, assessed via EHR and state mortality data. Importantly, patient and clinician stakeholders were engaged in the design and implementation of this study, and health care leaders helped design the intervention to minimize disruption of clinical workflows, maximizing sustainability should either intervention be found effective. Study results are expected in 2021.

In response to MHRN findings, health care systems began implementing a series of suicide prevention and intervention approaches as part of large-scale Zero Suicide initiatives (http://zerosuicide.edc.org). The Henry Ford Health System began this work in 2001, whereas Kaiser Permanente began evaluating varying models of Zero Suicide implementation across five regional health care systems in 2016 (7). Systems chose from a range of evidence-based interventions, including screening and assessment, safety planning, engagement in care, care management, caring contacts, follow-up after hospital or emergency discharge, means reduction, and intensive suicide risk treatment. A NIMH-funded grant has supported evaluation of the model at each system by using the normalization process theory framework. Researchers partner with leaders and clinicians to document the implemented interventions, develop metrics aligned with each approach, facilitate a learning collaborative across sites, and support ongoing improvement efforts.

Although PHQ-9 item 9 data were reasonably good at identifying patients at increased suicide risk, we adopted machine-learning methods to develop potentially more accurate suicide risk prediction models. Models use data from nearly 20 million visits from 2.96 million patients (58.3% [N=1,727,335] women, 0.9% [N=26,139] American Indian or Alaskan Native, 7.1% [N=209,094] Asian, 1.6% [N=47,113] Native Hawaiian/Pacific Islander, 8.3% [N=247,083] Black or African American, 59.0% [N=1,748,406] White, 25.5% [N=754,585] Hispanic, 0.8% [N=23,039] more than one race, 22.3% [N=659,993] unknown or not reported) to better consolidate multiple factors for suicide risk than use of the PHQ-9 alone (8). This method could allow health care systems to provide more intensive interventions for those at highest risk while opting for less intensive interventions for patients at lower risk. Importantly, and in contrast with results from PHQ-9 item 9, health care systems have the ability to set these risk thresholds to match available interventions and resources. Our models and others have a global classification accuracy of ≥80%, but given relatively low baseline rates of suicide attempts, positive predictive values (PPVs) are low, often below 0.01 (9). However, models with similar PPVs are widely used in other clinical areas. For example, many guidelines recommend statins for people with at least 10% risk of a cardiovascular event in the next 10 years. Similarly, using suicide risk models, we can accurately identify individuals with a 5% risk of a suicide attempt in the next 90 days. Essentially, the threshold for acceptability of a PPV depends on the balance of risks and harms with the indicated subsequent clinical actions. Most clinicians would agree that the risk of starting a statin is low and a reasonable secondary prevention strategy; the equivalent strategy for elevated suicide risk is not yet known but represents an opportunity for future research within this learning health system.

Researchers have been working with care delivery leaders to adopt suicide risk models for clinical use. At Kaiser Permanente Northern California, suicide risk models have been embedded in the EHR and run in the background (without display to clinicians), demonstrating similar model performance in this diverse external validation cohort. Researchers are now working with clinical leaders to determine an appropriate risk alert threshold and suitable workflows (10). At HealthPartners in Minnesota, suicide risk models are integrated with the EHR to produce weekly reports of members with serious mental illness or increased risk of hospitalization who are also at elevated risk for suicide. This approach prompts behavioral health case managers to complete CSSRS assessments and evaluate the need for more intensive interventions. Behavioral health clinicians at one outpatient clinic at Kaiser Permanente Washington are piloting use of a column in clinicians’ EHR calendars flagging patients with elevated suicide risk; clinicians are encouraged to have flagged patients complete the CSSRS.

Discussions between researchers and clinicians about these implementation strategies have led to shared recognition of the importance of research to understand how patients and clinicians interpret these risk models, how they experience conversations about suicide risk and prevention, and what clinical capacity is needed to respond to at-risk patients to inform future implementation strategies. A qualitative study at three health care systems uses the Consolidated Framework for Implementation Research to interview administrators, clinicians, case managers, patients, and insurance members to assess these needs. Interviews are conducted at different stages of preimplementation and implementation as an important step in improving patient-centered care and soliciting stakeholder perspectives about the appropriate uses and limitations of predictive modeling. Understanding the implementation context at various levels across organizations will inform future implementation strategies in other clinical settings.

A learning health care system where care informs research and research informs care hinges on ongoing relationships and shared priorities between health care system leaders, frontline clinicians, and researchers. These partnerships are built on trust that has evolved over many years of collaboration, shared common interests and goals, and frank conversations. To forge successful partnerships, in some cases researchers hear clinical concerns that are translatable to fundable research ideas. In other cases, researchers learn of research priorities that are translatable to care system priorities. Over time, researchers have become more embedded in clinical operations, and clinical leaders have become embedded members of the research teams. Bidirectional communication and collaboration between research and clinical staff require efforts on both sides to keep these partnerships viable and valuable to both groups.

As described in this column, successful learning health care systems do not undertake just one kind of research. A variety of research methods have been utilized, including observational studies, pragmatic clinical trials, implementation, machine learning, and qualitative research. This broad array of approaches allows researchers flexibility to tailor approaches to specific research or clinical questions, making use of existing data sources when possible and fitting into clinical workflows as necessary. Researchers need all of these tools to function effectively in learning health care systems and to better understand implementation issues across cultures, diverse patient populations, and varied clinical contexts.

The goals of health care systems and researchers are the same: to improve the care, outcomes, and experiences of patients. Conducting embedded research in a learning health care system not only provides opportunities to improve patient care more quickly than with traditional research models but also requires researchers to be flexible and adept at designing pragmatic research that minimizes clinical disruptions. Aligning health-care-system, funder, and research priorities is key to success.