Analysis of Hospitals Switching From a “Danger to Self” Question to Universal Columbia‐Suicide Severity Rating Scale Screening: Impact on Screenings, Identification of Suicide Risk, and Documented Psychiatric Care
Publication: Psychiatric Research and Clinical Practice
Abstract
Objective
Sutter Health launched system‐wide general population standardized suicide screening with the Columbia‐Suicide Severity Rating Scale (C‐SSRS) screen (triage) version in 23 hospitals in 2019, replacing a one‐question “danger to self” (DTS) assessment. This study analyzed the impact of C‐SSRS implementation on screening rates, positive screenings, and documented psychiatric care within 90 days for all patients and a subgroup diagnosed with Major Depressive Disorder (MDD).
Methods
Adults seen at hospitals in the pre‐period (July 1, 2017−June 30, 2019) and post‐period (July 1, 2019−December 31, 2020) were identified using electronic health records. Outcomes were compared using chi‐square statistics and interrupted time series (ITS) models.
Results
Pre‐period, 92.8% (740,984/798,653) of patients were screened by DTS versus 84.6% (504,015/595,915) by C‐SSRS in the post‐period. Positive screening rates were 1.5% pre‐period and 2.2% post‐period, and 9.2% pre‐period versus 10.8% post‐period for those with MDD. Among individuals with positive screenings, 64.0% (pre‐period) had documented follow‐up psychiatric care versus 52.5% post‐period and 66.4% of those with moderate or high‐risk. Among all patients seen there was an overall increase in documentation of psychiatric care within 90 days (0.87% pre‐ to 0.96% post‐period). ITS models revealed a 9.6% decline in screening, 1.3% increase in positive screenings, and 12.9% decline in documented psychiatric care following C‐SSRS implementation (all p < 0.01).
Conclusions
Following implementation, there was meaningful increase in suicide risk identification, and an increase in the proportion of patients with documented psychiatric care. Observed relative declines in screening warrant future research examining opportunities and barriers to general population C‐SSRS use.
Highlights
•
Twenty‐three hospitals switched from using a “danger to self” (DTS) question among the adult general population to standardized suicide screening with the Columbia‐Suicide Severity Rating Scale (C‐SSRS).
•
This change led to a decrease in the proportion of patients screened (92.8% to 84.6%) and a meaningful increase in rates of positive screening (with suicide risk) from 1.5% to 2.2%
•
Among all patients seen there was an overall increase in patients with subsequent documentation of psychiatric care within 90 days (0.87% pre‐ to 0.96% post‐period).
•
Among people screening positive for suicidality, there was a relative reduction in documented psychiatric care within 90 days from 64.0% to 52.5%, likely due to C‐SSRS identifying people with low risk suicidal ideation.
•
Switching from the unvalidated “DTS” question to using the validated C‐SSRS resulted in successfully identifying more patients at risk for suicide and the appropriate level of care needed.
Suicide is a pervasive and growing public health issue in the United States (1, 2), yet screening for suicide risk remains limited. Suicide risk screening is more common among individuals with mental health disorders who are known to have increased suicide risk (3, 4), with depressive and anxiety disorders being the most common among individuals who die by suicide (4). Major Depressive Disorder (MDD) has a high lifetime prevalence of 20.6% in the general population (5) and is associated with suicidality (6, 7). Coordinated efforts to improve earlier identification of people with suicide risk are urgently needed in order to connect them with appropriate care and prevent suicide deaths (8). Research indicates that 83% of individuals who died by suicide received healthcare in the year prior to their death (9, 10), yet many of these people are not being identified as at‐risk. These statistics represent a missed opportunity for intervention. Insufficient detection, monitoring, and follow‐up interventions in healthcare settings contribute to suicide deaths (11, 12, 13). Effective interventions such as safety planning and psychotherapy exist (14, 15), but are not consistently available.
Many people with suicide risk present for care in acute care settings, including emergency departments (EDs) (16, 17). The ED‐SAFE study demonstrated the effectiveness of universal suicide screening in EDs and brief interventions to reduce suicide attempts (18, 19). In 2018, The Joint Commission revised its National Patient Safety Goal requirements by standardizing suicide screening in behavioral health acute care settings and recommending use of validated suicide screening tools such as the C‐SSRS (20, 21). The C‐SSRS screen version (triage version) is a 6‐item questionnaire which detects suicide risk and severity and immediacy of suicide risk (22, 23), and its validity and feasibility has been established in emergency, psychiatric department, and general inpatient departments (24, 25, 26). While a number of health systems have adopted C‐SSRS screening (27), there is little mention of prior screening methods except recognition of the need for a standardized approach (22, 23). It is unknown whether screening with C‐SSRS compared to pre‐existing un‐validated screening questions has led to measurable changes in important population health outcomes.
Whereas the C‐SSRS is often administered based on evidence of suicide risk or existing mental health needs (24, 25, 28), this retrospective study examined the implementation of the C‐SSRS within the general population. Specifically, this study evaluated the impact of switching from a single “Danger to Self” (DTS) screening question to the C‐SSRS questionnaire for all patients seen in its 23 acute care hospitals with respect to three outcomes: rates of (1) screening, (2) positive screenings for suicide risk, and (3) documentation of follow‐up psychiatric care within the electronic health record (EHR).
Setting and Screening Methods
This research took place at Sutter Health, a large integrated healthcare delivery system in northern California which cares for approximately 3.5 million people each year in 100+ ambulatory clinics, 23 acute‐care hospitals, four acute care behavioral health centers, and 6 ambulatory behavioral health clinics. Prior to the C‐SSRS implementation, clinicians answered one question in the EHR about whether “DTS,” such as suicidal ideation or behavior or other self‐harm indicators were observed or expressed from a patient. DTS was assessed by RNs/clinicians and was required as part of the standard admission process across Sutter Health hospitals. DTS was based on clinical judgment and the wording of the questions asked and exactly when it was assessed varied across the hospitals.
Implementation of the C‐SSRS in these acute care hospitals and the population identified with suicide risk by C‐SSRS has been fully described elsewhere (29). Sutter Health first piloted use of the screen version (triage version) of the C‐SSRS in one general ED and two acute care behavioral health departments, then launched system‐wide standardized use of the C‐SSRS on July 1, 2019, in all acute care facilities, replacing the DTS question, and integrating C‐SSRS screening questions into the EHR. A workgroup of system stakeholders coordinated the implementation and met regularly. This group also organized standardized 20‐min C‐SSRS trainings conducted online or in person for approximately 9000 acute care registered nurses, including training on administration and entry in the EHR. The C‐SSRS is administered verbally, primarily by nurses, in EDs and inpatient acute care settings to all patients 10 years or older. With guidance from the developers of the C‐SSRS, individuals' responses were assigned to either low, moderate, or high risk categories (Online Supplement Table S1), along with accompanying practice recommendations. Low risk recommendations included considering mental health referrals, and for moderate or high risk included further assessment, immediate provider notification and mental health consultations, and additional safety precautions for high risk individuals.
METHODS
This EHR‐based observational cohort study analyzed changes in rates of screenings, positive screenings, and documented psychiatric care associated with the implementation of the C‐SSRS. The study period included a 24‐month pre‐launch “pre‐period” (July 1, 2017 to June 30, 2019) and 18‐month “post‐period” (July 1, 2019 to December 31, 2020). The cohort of patients seen included unique adults (age ≥18) in each time period with an index encounter at any of the 23 hospitals. For patients with multiple hospital encounters within the time period, their first encounter with a completed screening served as the index encounter, or if they were never screened, their first hospital encounter served as the index encounter. This study was reviewed and approved by Sutter Health's institutional review board.
Primary Outcomes
In the pre‐period, screening was measured by presence of response to the DTS question, and it was not possible to measure level of suicide risk. In the post‐period, screening was measured by presence of a complete C‐SSRS questionnaire. If patients had multiple screenings during a hospitalization, the first complete screening was used. Positive screening was defined in the pre‐period as having DTS identified and in the post‐period as being identified with low, moderate, or high risk by C‐SSRS.
A composite variable captured EHR documentation of any psychiatric care within 90 days of the index encounter, including transfer to psychiatric unit, discharge to psychiatric hospital, or behavioral health consultation/referral. Transfers and discharges to both Sutter and non‐Sutter hospitals were retrieved from hospital discharge data. Psychiatric consultations were limited to care within Sutter's acute and ambulatory care system. Referrals to psychiatric care were predominately within Sutter's system, with the exception of referrals to a vendor responsible for coordinating behavioral health for ambulatory care patients in specific geographic areas. Other subsequent care measured included patient hospitalization and length of stay at the time of the index encounter, and additional hospitalizations with mental health diagnoses recorded, behavioral health acute care hospitalizations, ED visits, and C‐SSRS or DTS screenings recorded.
Covariates
Other measures included patient information retrieved from the EHR in the 12 months prior to and including the index date. Sociodemographic characteristics included age, sex, race, ethnicity, language spoken, marital status, median household income for patient's postal code, and insurance type. Healthcare utilization included type of encounter (ED, inpatient, or observation), department of index encounter, and number of prior primary care and ED encounters. Clinical characteristics included diagnosis of type 2 diabetes, hypertension, cancer, congestive heart failure, chronic pulmonary disease, and Charlson Comorbidity Index of 1 or more (30). Mental health diagnoses included MDD, anxiety disorder, depressive disorders, substance abuse disorder, bipolar spectrum disorder, schizophrenia spectrum disorder, Attention‐Deficit/Hyperactivity Disorder, autism, gender dysphoria, dementia, eating disorder, conduct/disruptive disorder, personality disorder, and were defined with International Classification of Diseases, Tenth Revision (ICD‐10) codes used in the Mental Health Research Network (31, 32). Prior suicide ideation was measured by the presence of ICD‐10 codes (Online Supplement Table S2).
Statistical Analysis
Descriptive statistics were used to describe population characteristics and primary outcomes in the pre‐period and post‐period. Subgroup analyses were conducted for patients with: (1) evidence of MDD, and (2) moderate or high risk identified by C‐SSRS. Patients with C‐SSRS moderate or high risk were grouped for analysis based on the logic that they most urgently need psychiatric intervention, and individuals identified with low suicide risk via C‐SSRS would be unlikely to screen positive using the DTS question which was intended to capture immediate danger. Data were analyzed using standard tests (Wilcoxon rank‐sum tests for continuous variables and χ2 tests for categorical variables) with an alpha of 0.05 (two‐sided) as the level of significance. Analyses were performed using SAS version 9.4.
As this study analyzed simultaneous implementation of the C‐SSRS at all hospitals without randomization, interrupted time series (ITS) analyses were conducted on monthly outcomes from July 2017 to December 2020 to assess the longitudinal effects of the C‐SSRS implementation on the primary outcomes. ITS considered an expected trend during the pre‐period and the trend observed in the post‐period and identified changes in the trend between time periods to evaluate the effectiveness of C‐SSRS implementation (33, 34). Scatter plots of the time series were also created to visualize trends and seasonal patterns.
Multiple ITS analyses were conducted for each outcome and for each group. For the outcome Screening Rate, two models were run separately for the overall patients seen and for the subgroup with MDD diagnosis. For the outcome Positive Screening Rate, one model was created for comparing the rate of positive screenings among patients screened, another for the MDD subgroup who were screened, and one model for positive DTS screening pre‐period and C‐SSRS moderate or high risk post period among patients screened. For the outcome Documented Psychiatric Care Rate, three separate models were run for all patients identified with positive screenings, those with positive DTS screening pre‐period and C‐SSRS moderate or high risk post‐period, and those with MDD and positive screenings.
RESULTS
798,653 unique individuals were seen in the pre‐period and 595,915 in the post‐period (average per month: 33,277 pre‐ vs. 33,106 post‐period) (Table 1). The population (pre vs. post) had a mean age of 48.8 versus 48.7, was 57.2% versus 56.5% female, and both time periods had similar distributions by race and ethnicity: Non‐Hispanic White 49.8% versus 48.3%, Non‐Hispanic Black 11.3% versus 11.3%, Hispanic 21.9% versus 23.0%, and Asian 9.6% versus 9.4%. Patient characteristics and healthcare utilization were similar in both time periods.
Pre‐period: July 1, 2017 to June 30, 2019 N = 798,653 | Post‐period: July 1, 2017 to December 31, 2020 N = 595,915 | ||||
|---|---|---|---|---|---|
| Mean | SD | Mean | SD | p‐value | |
| Age | 48.8 | 20.3 | 48.7 | 20.3 | 0.02 |
| N | % | N | % | ||
|---|---|---|---|---|---|
| Number of months | 24 | 18 | |||
| Sexb | <0.001 | ||||
| Male | 341,937 | 42.8 | 259,447 | 43.5 | |
| Female | 456,690 | 57.2 | 336,424 | 56.5 | |
| Race/ethnicity | <0.001 | ||||
| Non‐Hispanic White | 397,457 | 49.8 | 287,717 | 48.3 | |
| Non‐Hispanic Black | 90,053 | 11.3 | 67,506 | 11.3 | |
| Hispanic | 174,700 | 21.9 | 137,110 | 23.0 | |
| Asian | 76,956 | 9.6 | 56,221 | 9.4 | |
| Other/unknown | 59,587 | 7.5 | 47,361 | 7.9 | |
| Language spoken | <0.001 | ||||
| English | 705,888 | 88.4 | 526,352 | 88.3 | |
| Spanish | 54,719 | 6.9 | 42,391 | 7.1 | |
| Other/unknown | 38,046 | 4.8 | 27,172 | 4.6 | |
| Marital status | <0.001 | ||||
| Married | 320,717 | 40.2 | 235,186 | 39.5 | |
| Divorced/single | 379,089 | 47.5 | 287,598 | 48.3 | |
| Other/unknown | 98,847 | 12.4 | 73,131 | 12.3 | |
| Median household income (by postal code)b | <0.001 | ||||
| <$50,000 | 199,742 | 25.0 | 148,614 | 24.9 | |
| $50,000–$99,999 | 468,496 | 58.7 | 349,595 | 58.7 | |
| >$100,000 | 111,495 | 14.0 | 82,172 | 13.8 | |
| Insurance | <0.001 | ||||
| Commercial | 229,186 | 28.7 | 164,351 | 27.6 | |
| Medicaid/Medi‐Cal | 89,845 | 11.2 | 65,647 | 11.0 | |
| Medicare FFS/HMO | 234,622 | 29.4 | 165,684 | 27.8 | |
| Other (multiple, self, missing) | 245,000 | 30.7 | 200,233 | 33.6 | |
| Department of index encounter | <0.001 | ||||
| Emergency medicine | 702,781 | 88.0 | 528,287 | 88.7 | |
| Psychiatry/psychology | 1624 | 0.2 | 890 | 0.1 | |
| Obstetrics and gynecology | 48,653 | 6.1 | 35,831 | 6.0 | |
| Other acute care | 41,007 | 5.1 | 28,748 | 4.8 | |
| Prior primary care encounters | 141,309 | 17.7 | 109,246 | 18.3 | <0.001 |
| Prior ED encounters | 94,681 | 11.9 | 89,113 | 15.0 | <0.001 |
| Clinical characteristics | |||||
| CCI 1+ | 283,034 | 35.4 | 189,710 | 31.8 | <0.001 |
| Diabetes (type 2) | 99,757 | 12.5 | 66,225 | 11.1 | <0.001 |
| Hypertension | 215,257 | 27.0 | 134,820 | 22.6 | <0.001 |
| Cancer | 36,041 | 4.5 | 27,782 | 4.7 | <0.001 |
| Congestive heart failure | 44,700 | 5.6 | 34,894 | 5.9 | <0.001 |
| Chronic pulmonary disease | 114,107 | 14.3 | 66,789 | 11.2 | <0.001 |
| Any mental health diagnosis | 104,978 | 13.1 | 86,873 | 14.6 | <0.001 |
| MDD diagnosis on or prior to index date | 28,412 | 3.6 | 18,638 | 3.1 | <0.001 |
| Anxiety disorder | 38,873 | 4.9 | 34,025 | 5.7 | <0.001 |
| Bipolar spectrum disorder | 8023 | 1 | 6656 | 1.1 | <0.001 |
| Depressive disorder | 33,098 | 4.1 | 28,085 | 4.7 | <0.001 |
| Schizophrenia spectrum disorder | 4880 | 0.6 | 4406 | 0.7 | <0.001 |
| Substance abuse disorder | 24,894 | 3.1 | 22,719 | 3.8 | <0.001 |
| Otherc | 15,535 | 1.9 | 12,910 | 2.2 | <0.001 |
| Suicide ideation prior to index visit | 3558 | 0.4 | 3388 | 0.6 | <0.001 |
a
C‐SSRS, Columbia‐Suicide Severity Rating Scale; CCI, Charlson Comorbidity Index; FFS, Fee‐For‐Service; HMO, Health Maintenance Organization; MDD, Major Depressive Disorder.
b
Missing values not reported in table.
c
Other mental health diagnoses included: ADHD, autism, gender dysphoria, dementia, eating disorder, conduct/disruptive disorder, personality disorder.
Changes After C‐SSRS Implementation
Screening decreased from 92.8% (740,984 out of 798,653) of patients using the DTS question (pre‐period) to 84.6% (504,015 out of 595,915) using the C‐SSRS (post‐period) (Table 2). The rate of screening positive increased from 1.5% of patients screened in the pre‐period to 2.2% post‐period (p < 0.001), with 1.3% identified as moderate or high risk by C‐SSRS. The overall proportion of patients screening positive out of all patients seen increased from 1.35% (10,791 out of 798,653) in the pre‐period to 1.82% (10,866 out of 595,915) in the post‐period.
| All patients | MDD subgroup | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Pre‐period (July 1, 2017 to June 30, 2019) | Post‐period (July 1, 2019 to December 31, 2020) | Pre‐period (July 1, 2017 to June 30, 2019) | Post‐period (July 1, 2019 to December 31, 2020) | |||||||
| N | % | N | % | p‐value | N | % | N | % | p‐value | |
| Number of months | 24 | 18 | 24 | 18 | ||||||
| Patients seen | ||||||||||
| Number of patients seen | 798,653 | 595,915 | 53,864 | 41,652 | ||||||
| Patients screened | ||||||||||
| Number of patients screened | 740,984 | 92.8 | 504,015 | 84.6 | <0.0001 | 51,384 | 95.4 | 37,345 | 89.7 | <0.001 |
| Patients screening positive (among patients screened) | ||||||||||
| Number of patients screening positive | 10,791 | 1.5 | 10,866 | 2.2 | <0.0001 | 4982 | 9.2 | 4489 | 10.8 | <0.001 |
| Number of patients at low risk | n/a | 4086 | 0.9 | n/a | n/a | 1195 | 2.9 | n/a | ||
| Number of patients at moderate or high risk | n/a | 6780 | 1.3 | n/a | n/a | 3294 | 7.9 | n/a | ||
| Any documented follow‐up psychiatric care (composite of 1, 2, 3) | 6910 | 64.0 | 5706 | 52.5 | <0.0001 | 3518 | 70.6 | 3035 | 67.6 | 0.002 |
| 1. Transfer to psych unit or discharge to psych hospital (Day 0) | 4343 | 40.2 | 3390 | 31.2 | <0.0001 | 2015 | 40.4 | 1747 | 38.9 | 0.129 |
| 2. Behavioral health referral (Day 0–90) | 3309 | 30.7 | 3044 | 28.0 | <0.0001 | 1664 | 33.4 | 1588 | 35.4 | 0.04 |
| 3. Behavioral health consult/encounter visits (Day 0–90) | 2943 | 27.3 | 2507 | 23.1 | <0.0001 | 1899 | 38.1 | 1589 | 35.4 | 0.006 |
| Care received at index encounter | ||||||||||
| Emergency department visit | 7972 | 73.9 | 7860 | 72.3 | 0.01 | 3309 | 66.4 | 3082 | 68.7 | 0.02 |
| Hospitalization (inpatient) | 2618 | 24.3 | 2775 | 25.5 | 0.03 | 1595 | 32.0 | 1325 | 29.5 | 0.009 |
| Hospitalization length of stay (LOS): Mean (SD) | 6.5 | 11.1 | 5.7 | 8.3 | <0.0001 | 6.2 | 12.0 | 6.1 | 7.3 | 0.12 |
| Other subsequent hospital care | ||||||||||
| Hospitalizations with mental health diagnosis (Day 0–90) | 3931 | 36.4 | 4098 | 37.7 | 0.05 | 2237 | 44.9 | 1963 | 43.7 | 0.25 |
| Emergency department visit (Day 1–90) | 2219 | 20.6 | 2517 | 23.2 | <0.0001 | 956 | 19.2 | 971 | 21.6 | 0.003 |
| Additional suicide risk screening (Day 1–90) | 6069 | 56.2 | 5241 | 48.2 | <0.0001 | 3246 | 65.2 | 2529 | 56.3 | <0.001 |
a
MDD, Major Depressive Disorder.
b
Behavioral health consult/encounter includes inpatient behavioral health acute care and ambulatory care encounters in behavioral health.
During the pre‐period, 64.0% of patients who screened positive had documentation of psychiatric care within 90 days compared to 52.5% (p < 0.001) in the post‐period. Among patients screening positive, rates of transfer or discharge to acute psychiatric care decreased from 40.2% pre‐period to 31.2% post‐period (p < 0.001), rates of referrals to behavioral health providers declined from 30.7% pre‐period to 28% of patients post‐period (p < 0.001), and behavioral health consultations declined from 27.3% to 23.1% (p < 0.001). However, out of all patients seen at the hospital, documentation of psychiatric care within 90 days increased from 0.87% pre‐period (6910 out of 798,653 patients) to 0.96% in the post‐period (5706 out of 595,915 patients).
When comparing patients screening positive by DTS in the pre‐period and patients screening positive with C‐SSRS moderate or high risk in the post‐period, rates of documented psychiatric care increased from 64.0% to 66.4% (p = 0.001), transfers/discharges to acute psychiatric care increased 40.2% to 42.4% (p = 0.005), behavioral health consultations increased from 27.3% to 29.7% (p < 0.001) and referrals to behavioral health providers from 30.7% to 34.4% (p < 0.001) (Additional file 4).
There were 6.7% (53,864 out of 798,653) patients in the pre‐period and 7.0% (41,652 out of 595,915) in the post‐period with a documented MDD diagnosis. Rates of screening, screening positive, and psychiatric care were higher in the MDD subgroup compared to the overall cohort. From pre‐ to post‐period screening of patients with MDD decreased, 95.4% to 89.7% (p < 0.0001), positive screenings increased from 9.2% to 10.8% (p < 0.001), and documented psychiatric care for those screening positive decreased from 70.6% to 67.6% (p = 0.002).
Interrupted Time Series Models
The scatter plot of monthly screening rates indicated a slight monthly increasing trend during the pre‐period but an immediate drop and a slight monthly declining trend in the post‐period following C‐SSRS implementation (Online Supplement Figure S1). The ITS model further supported this evidence showing a decrease of −9.61% (p < 0.01) in the first month after the C‐SSRS was implemented with a monthly trend change of −0.34% (p < 0.01) compared to the pre‐period trend (Table 3). For the subgroup diagnosed with MDD, ITS results showed an immediate decrease of screening rates by −9.4% (p < 0.01) after C‐SSRS implementation, followed by a monthly trend change of −0.19% (p < 0.01) relative to the pre‐period trend.
| Coefficient | ||||
|---|---|---|---|---|
| Outcome | ß0 | ß1 | ß2 | ß3 |
| Pre‐period baseline level | Pre‐period trend | Post‐period immediate level change | Post‐period trend change | |
| Screening, % | ||||
| Overall | 88.35** | 0.16** | −9.61** | −0.34** |
| MDD subgroup | 89.9** | 0.17** | −9.4** | −0.19* |
| Positive screening, % | ||||
| Overall | 2.13** | −0.01 | 1.29** | −0.02 |
| Moderate or high risk | 2.13** | −0.01** | 0.04 | 0.01 |
| MDD subgroup | 9.8** | −0.10** | 6.09** | −0.14** |
| Documented follow‐up psychiatric care within 90 days, % | ||||
| Overall | 57.48** | 0.14' | −12.86** | 0.14 |
| Moderate or high risk | 57.48** | 0.14* | 4.0** | −0.34** |
| MDD subgroup | 62.69** | 0.32** | −8.57 | −0.16 |
a
MDD, Major Depressive Disorder.
b
The following segmented linear regression was used for ITS analyses: Y = ß0 + ß1 T + ß2 X + ß3 P. The ITS model required four variables: T is the months elapsed since the start of the study, X is a dummy variable indicating the pre‐period (X = 0) or the post‐period (X = 1), P is the months elapsed since the C‐SSRS implementation (p = 0 for pre‐period), and Y is the monthly outcome. Model parameters included ß0 representing the baseline level at T = 0, ß1 representing the underlying pre‐period trend, ß2 indicating the level change following the C‐SSRS implementation, and ß3 indicating the slope change following the C‐SSRS implementation.
'p < 0.1, *p < 0.05, **p < 0.01.
The scatter plot of monthly rates of positive screenings showed a stable trend during the pre‐period, but an immediate increase after C‐SSRS implementation followed by a subsequent declining trend (Figure 1). The proportion of patients screening positive increased by 1.29% (p < 0.01) in the month following implementation, followed by a slight decrease of −0.02% in the monthly trend compared to the pre‐period trend. For the proportion of patients identified with moderate or high risk in the post‐period, the immediate effect was a slight increase of 0.04%. For patients with diagnosed MDD, the ITS model showed an increase of 6.09% (p < 0.01) in the first month following implementation, relative to the pre‐trend, with a decrease of −0.14% (p < 0.01) in the monthly trend thereafter.
Among patients screening positive, the monthly rate of documented follow‐up psychiatric care appeared stable during pre‐period but varied across different risk levels identified by C‐SSRS during the post‐period (Figure 2). The ITS analyses showed a decrease of −12.86% (p < 0.01) in overall patients with documented psychiatric care in the month following C‐SSRS implementation followed by a similar monthly trend as the pre‐period. However, for patients identified with moderate or high risk in post‐period, the ITS model showed an immediate increase of 4.0% (p < 0.01) followed by a monthly trend decrease of −0.34% (p < 0.01). For the MDD subgroup, there was an immediate decrease of −8.57% in psychiatric care after C‐SSRS implementation with a monthly trend decrease of −0.16%.
DISCUSSION
This study analyzed changes in rates of screening, positive screenings for suicide risk, and documented psychiatric care in 23 hospitals that implemented the C‐SSRS. To our knowledge, this is the first study to assess the impact of switching from a “DTS” question to a standardized and validated suicide screening approach. We found an association between C‐SSRS implementation and a decrease in screening rates (92.8% to 84.6%); a meaningful increase in the proportion of screened patients screening positive (1.5% to 2.2%); and among those screening positive a decrease in the proportion of patients with documentation of psychiatric care within 90 days (64.0% to 52.5%), but an increase in documentation of psychiatric care within 90 days among all patients seen (0.87% to 0.96%). Similar trends were observed for the subgroup of patients with MDD. A subgroup analysis compared patients screening positive using the DTS question to those identified with moderate or high risk by the C‐SSRS, based on the logic that these are the groups warranting immediate psychiatric care, and found a slight increase in psychiatric care (64.0% to 66.4%).
The observed decrease in screening rates may be explained by the additional time required for screening using the C‐SSRS. Additionally, some patients may have been unable or unwilling to respond to the C‐SSRS. This analysis used only records of completed C‐SSRS screening, excluding records with incomplete and partial C‐SSRS data. Future qualitative research is required to understand clinician and patient experience with the C‐SSRS and the specific barriers with integrating it into general population clinical workflows.
Systematic screening for suicide risk is a key ingredient of health system interventions such as the Zero Suicide Initiative (27), and with the C‐SSRS, we observed a meaningful increase in positive screenings. This increase may have several potential explanations. First, the C‐SSRS is a more valid and reliable instrument for identifying individuals with suicide risk compared to the single DTS question, and it can determine suicide risk severity (22, 23). Second, the “DTS” question was focused on immediate risk of self‐harm and could have excluded people with low risk. Comparing people identified with DTS with those identified as moderate or high risk with the C‐SSRS, we observed a change from 1.5% screening positive by DTS to 1.3% screening positive with moderate or high risk by C‐SSRS. Third, the COVID‐19 pandemic may have influenced rates of suicide risk (35). A recent analysis in the same population found that COVID‐19 led to a 19% reduction in patients seen in these hospitals, but an increase in those identified as moderate or high risk. Policies such as “shelter‐in‐place” orders may have discouraged utilization of healthcare services for non‐urgent and low‐acuity issues (36).
Considering the overall increase in the proportion of patients with documented psychiatric care out of all patients seen, the decrease in the proportion of patients with positive screenings with documentation of psychiatric care may be partially explained by the fact that the post‐period positive screening cohort includes people identified with low risk. Measures of psychiatric care were primarily inpatient based and may not be appropriate for those with low risk. One advantage of the C‐SSRS is its suicide risk categorization and consequently the ability to direct people with lower risk to outpatient care. When comparing people screening positive by DTS with those moderate or high risk by C‐SSRS, there is a slight increase in rates of psychiatric care, indicating that follow‐up care was prioritized for those at higher risk.
Limitations
The generalizability of this study may be limited due to its focus on adults in one health system. Ideally the study would have incorporated data on suicide attempts and deaths, but these data were not available. This study was unable to distinguish whether mental health or suicide risk was the primary reason for a patient's hospital visit. This study may overestimate follow‐up psychiatric care as our measure included referrals to behavioral health providers, which may not have resulted in an encounter. This study may also underestimate psychiatric care and diagnoses. Much psychiatric care is difficult to access and provided in private practices (37, 38), so those encounters were not documented in this health system's EHR, however these limitations were present in both pre‐ and post‐period so likely did not influence results. Also documentation may not perfectly reflect practice, some screenings may not have been documented, and some documented screenings may not have been asked verbally to the patient. These constraints with documentation and interoperability of mental health information are a critical challenge to research, clinical care, and population health. Finally, this study compared a “DTS” question which measured potential for self‐harm to a suicide risk questionnaire. Despite different screening goals, self‐harm often leads to increased suicide risk and both populations warrant appropriate psychiatric care (39, 40).
CONCLUSION
These findings present evidence that switching from the unvalidated “DTS” question to using the validated C‐SSRS resulted in successfully identifying more patients at risk for suicide and the appropriate level of care needed. Future research may be necessary to examine the experience of clinical staff and patients with the C‐SSRS and to understand barriers to increasing its use. Standardized suicide screenings, if successfully adopted in healthcare settings nationwide, have the potential to efficiently identify people at risk for suicide, providing an important opportunity for prevention.
Footnotes
Statement of originality: The authors attest and affirm that this manuscript is an original work that has not been submitted to nor published anywhere else.
Previous presentations: None.
We would like to acknowledge and thank the many clinical teams who worked to implement the Columbia Suicide Severity Rating Scale (C‐SSRS) across Sutter Health. We thank and acknowledge Pragati Kenkare for her careful stewardship of the data in this manuscript. We thank Sylvia Sudat for advice on the statistical modeling. We thank ApotheCom (Yardley, PA) for editorial assistance which was funded by Janssen Scientific Affairs, LLC. This study was funded by Janssen Scientific Affairs, LLC.
ED, QH, SD, ML, TN, AK, and KA are/were employed by Sutter Health. ED is currently employed by University of Connecticut. DB is employed by University of California, San Francisco. ED, QH, SD, ML, and KA received research funding support from Janssen Scientific Affairs to conduct this study. JP is a Janssen employee and Johnson & Johnson stockholder.
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
This research was approved by the Sutter Health organizational institutional review board with an authorized waiver of consent. All research was carried out in accordance with relevant guidelines and regulations.
Consent for publication: Not applicable.
Supplementary Material
Supplementary Material
- Download
- 96.79 KB
REFERENCES
1.
Centers for Disease Control and Prevention . Web‐based Injury Statistics Query and Reporting System (WISQARS). Leading causes of death visualization tool [Internet]. Centers for Disease Control and Prevention, National Center for Injury Prevention and Control. 2021 [cited March 19, 2021]. Available from: https://wisqars‐viz.cdc.gov:8006/lcd/home
2.
Stone DMJC, Mack KA. Changes in suicide rates — United States, 2018–2019. MMWR (Morb Mortal Wkly Rep). 2021;70(8):261–268. https://doi.org/10.15585/mmwr.mm7008a1
3.
Harris EC, Barraclough B. Suicide as an outcome for mental disorders. A meta‐analysis. Br J Psychiatr. 1997;170(3):205–228. https://doi.org/10.1192/bjp.170.3.205
4.
Yeh H.‐H, Westphal J, Hu Y, Peterson EL, Williams LK, Prabhakar D, et al. Diagnosed mental health conditions and risk of suicide mortality. Psychiatr Serv. 2019;70(9):750–757. https://doi.org/10.1176/appi.ps.201800346
5.
Hasin DS, Sarvet AL, Meyers JL, Saha TD, Ruan WJ, Stohl M, et al. Epidemiology of adult DSM‐5 major depressive disorder and its specifiers in the United States. JAMA Psychiatr. 2018;75(4):336–346. https://doi.org/10.1001/jamapsychiatry.2017.4602
6.
Angst J, Angst F, Stassen HH. Suicide risk in patients with major depressive disorder. J Clin Psychiatr. 1999;60(Suppl 2):57–62. Discussion 75‐6, 113‐6.
7.
Coryell W, Young EA. Clinical predictors of suicide in primary major depressive disorder. J Clin Psychiatr. 2005;66(4):412–417. https://doi.org/10.4088/jcp.v66n0401
8.
Office of the Surgeon G, National Action Alliance for Suicide P . Publications and reports of the surgeon general. 2012 National Strategy for Suicide Prevention: Goals and objectives for action: a report of the US Surgeon general and of the National Action Alliance for Suicide Prevention. Washington: US Department of Health & Human Services (US); 2012.
9.
Ahmedani BK, Simon GE, Stewart C, Beck A, Waitzfelder BE, Rossom R, et al. Health care contacts in the year before suicide death. J Gen Intern Med. 2014;29(6):870–877. https://doi.org/10.1007/s11606-014-2767-3
10.
Stuck AR, Wilson MP, Chalmers CE, Lucas J, Sarkin A, Choi K, et al. Health care usage and suicide risk screening within 1 year of suicide death. J Emerg Med. 2017;53(6):871–879. https://doi.org/10.1016/j.jemermed.2017.06.033
11.
Labouliere CD, Vasan P, Kramer A, Brown G, Green K, Rahman M, et al. “Zero Suicide” – a model for reducing suicide in United States behavioral healthcare. Suicidologi. 2018;23(1):22–30. https://doi.org/10.5617/suicidologi.6198
12.
U.S. Surgeon General . Surgeon General's Call to Action: to implement the national strategy for suicide prevention. 2021. U.S. Surgeon General, National Action Alliance for Suicide Prevention. https://www.hhs.gov/sites/default/files/sprc‐call‐to‐action.pdf
13.
Stanley B, Mann JJ. The need for innovation in health care systems to improve suicide prevention. JAMA Psychiatr. 2020;77(1):96–98. https://doi.org/10.1001/jamapsychiatry.2019.2769
14.
Inagaki M, Kawashima Y, Yonemoto N, Yamada M. Active contact and follow‐up interventions to prevent repeat suicide attempts during high‐risk periods among patients admitted to emergency departments for suicidal behavior: a systematic review and meta‐analysis. BMC Psychiatr. 2019;19(1):1–11. https://doi.org/10.1186/s12888-019-2017-7
15.
Melhem NM, Brent D. Do brief preventive interventions for patients at suicide risk work? JAMA Psychiatr. 2020;77(10):997–999. https://doi.org/10.1001/jamapsychiatry.2020.1287
16.
Ting SA, Sullivan AF, Boudreaux ED, Miller I, Camargo CA, Jr. Trends in US emergency department visits for attempted suicide and self‐inflicted injury, 1993–2008. Gen Hosp Psychiatr. 2012;34(5):557–565. https://doi.org/10.1016/j.genhosppsych.2012.03.020
17.
Owens PL, Mutter R, Stocks C. Mental health and substance abuse‐related emergency department visits among adults, 2007: statistical brief #92. Healthcare Cost and Utilization Project (HCUP) Statistical Briefs. Rockville: Agency for Healthcare Research and Quality; 2006.
18.
Boudreaux ED, Camargo CA, Jr, Arias SA, Sullivan AF, Allen MH, Goldstein AB, et al. Improving suicide risk screening and detection in the emergency department. Am J Prev Med. 2016;50(4):445–453. https://doi.org/10.1016/j.amepre.2015.09.029
19.
Miller IW, Camargo CA, Jr., Arias SA, Sullivan AF, Allen MH, Goldstein AB, et al. Suicide prevention in an emergency department population: the ED‐SAFE study. JAMA Psychiatr. 2017;74(6):563–570. https://doi.org/10.1001/jamapsychiatry.2017.0678
20.
The Joint Commission . National patient safety goal on suicide prevention applicable to critical access hospitals in July. The Joint commission perspectives: the official newsletter of the Joint Commission. 2019;39(12). https://www.jointcommission.org/‐/media/tjc/documents/resources/patient‐safety‐topics/suicide‐prevention/jcp1219.pdf
21.
The Joint Commission . National patient safety goal for suicide prevention. R3 Report: Requirement, rationale, reference. 2019(18). https://www.jointcommission.org/‐/media/tjc/documents/standards/r3‐reports/r3_18_suicide_prevention_hap_bhc_cah_11_4_19_final1.pdf
22.
Posner K, Brown GK, Stanley B, Brent DA, Yershova KV, Oquendo MA, et al. The Columbia–Suicide Severity Rating Scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatr. 2011;168(12):1266–1277. https://doi.org/10.1176/appi.ajp.2011.10111704
23.
Interian A, Chesin M, Kline A, Miller R, St Hill L, Latorre M, et al. Use of the Columbia‐Suicide Severity Rating Scale (C‐SSRS) to classify suicidal behaviors. Arch Suicide Res. 2018;22(2):278–294. https://doi.org/10.1080/13811118.2017.1334610
24.
Bjureberg J, Dahlin M, Carlborg A, Edberg H, Haglund A, Runeson B. Columbia‐Suicide Severity Rating Scale Screen Version: initial screening for suicide risk in a psychiatric emergency department. Psychol Med. 2021;52(16):1–9. https://doi.org/10.1017/s0033291721000751
25.
Katz I, Barry CN, Cooper SA, Kasprow WJ, Hoff RA. Use of the Columbia‐suicide severity rating Scale (C‐SSRS) in a large sample of Veterans receiving mental health services in the Veterans health administration. Suicide Life‐Threatening Behav. 2020;50(1):111–121. https://doi.org/10.1111/sltb.12584
26.
Roaten K, Johnson C, Genzel R, Khan F, North CS. Development and implementation of a universal suicide risk screening program in a safety‐net hospital system. Joint Comm J Qual Patient Saf. 2018;44(1):4–11. https://doi.org/10.1016/j.jcjq.2017.07.006
27.
Coleman KJ, Stewart CC, Bruschke C, Flores JP, Altschuler A, Beck A, et al. Identifying people at risk for suicide: implementation of screening for the Zero suicide initiative in large health systems. Adv Psychiatr Behav Health. 2021;1(1):67–76. https://doi.org/10.1016/j.ypsc.2021.05.016
28.
Brown LA, Boudreaux ED, Arias SA, Miller IW, May AM, Camargo CA, Jr, et al. C‐SSRS performance in emergency department patients at high risk for suicide. Suicide Life‐Threatening Behav. 2020;50(6):1097–1104. https://doi.org/10.1111/sltb.12657
29.
Dillon EC, Huang Q, Deng S, Li M, de Vera E, Pesa J, et al. Implementing universal suicide screening in a large healthcare system’s hospitals: rates of screening, suicide risk, and documentation of subsequent psychiatric care. Transl Behav Med. 2023;13(4):193–205. https://doi.org/10.1093/tbm/ibac117
30.
Quan H, Sundararajan V, Halfon P, Fong A, Burnand B, Luthi JC, et al. Coding algorithms for defining comorbidities in ICD‐9‐CM and ICD‐10 administrative data. Med Care. 2005;43(11):1130–1139. https://doi.org/10.1097/01.mlr.0000182534.19832.83
31.
Mental Health Research Network . Public resources from the mental health research Network. https://github.com/MHResearchNetwork/MHRN‐Central (2021). Accessed 18 Mar 2024.
32.
Coleman KJ, Stewart C, Waitzfelder BE, Zeber JE, Morales LS, Ahmed AT, et al. Racial‐ethnic differences in psychiatric diagnoses and treatment across 11 health care systems in the mental health research network. Psychiatr Serv. 2016;67(7):749–757. https://doi.org/10.1176/appi.ps.201500217
33.
Bernal JL, Cummins S, Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int J Epidemiol. 2017;46(1):348–355.
34.
Kontopantelis E, Doran T, Springate DA, Buchan I, Reeves D. Regression based quasi‐experimental approach when randomisation is not an option: interrupted time series analysis. BMJ. 2015;350(jun09 5):h2750. https://doi.org/10.1136/bmj.h2750
35.
Hill RM, Rufino K, Kurian S, Saxena J, Saxena K, Williams L. Suicide ideation and attempts in a pediatric emergency department before and during COVID‐19. Pediatrics. 2021;147(3). https://doi.org/10.1542/peds.2020-029280
36.
Hartnett KP, Kite‐Powell A, DeVies J, Coletta MA, Boehmer TK, Adjemian J, et al. Impact of the COVID‐19 pandemic on emergency department visits—United States, January 1, 2019–May 30, 2020. MMWR (Morb Mortal Wkly Rep). 2020;69(23):699–704. https://doi.org/10.15585/mmwr.mm6923e1
37.
Coombs NC, Meriwether WE, Caringi J, Newcomer SR. Barriers to healthcare access among US adults with mental health challenges: a population‐based study. SSM‐Popul Health. 2021;15:100847. https://doi.org/10.1016/j.ssmph.2021.100847
38.
Abar B, Holub A, Lee J, DeRienzo V, Nobay F. Depression and anxiety among emergency department patients: utilization and barriers to care. Acad Emerg Med. 2017;24(10):1286–1289. https://doi.org/10.1111/acem.13261
39.
Skegg K. Self‐harm. Lancet. 2005;366(9495):1471–1483. https://doi.org/10.1016/s0140-6736(05)67600-3
40.
Cooper J, Kapur N, Webb R, Lawlor M, Guthrie E, Mackway‐Jones K, et al. Suicide after deliberate self‐harm: a 4‐year cohort study. Am J Psychiatr. 2005;162(2):297–303. https://doi.org/10.1176/appi.ajp.162.2.297
Information & Authors
Information
Published In
History
Received: 15 November 2023
Revision received: 27 February 2024
Accepted: 4 March 2024
Published online: 3 April 2024
Published in print: Summer 2024
Authors
Funding Information
Janssen Scientific Affairs, LLC
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).