The Relationship Between Suicidal Behaviors and Zero Suicide Organizational Best Practices in Outpatient Mental Health Clinics
Abstract
Objective:
This study tested the hypothesis that fidelity of clinics to Zero Suicide (ZS) organizational practices is inversely related to suicidal behaviors of patients under clinical care.
Methods:
Using cross-sectional analyses, the authors examined the fidelity of 110 outpatient mental health clinics to ZS organizational best practices and suicidal behaviors of clinic patients in the year before a large-scale Zero Suicide implementation. Fidelity to ZS organizational best practices was assessed over a 1-year period with an adapted version of the ZS Organizational Self-Study instrument (17 items self-rated on a Likert scale of 1–5). Suicidal behaviors of patients were identified by extracting information on suicide attempts and deaths from a mandated statewide incident-reporting system database. Clinics were dichotomized into any or no suicide incidents during the year of observation. Logistic regression analyses were used to adjust for clinic census and population type (majority child or adult).
Results:
The clinics (N=110) served 30,257 patients per week. Clinics’ total average fidelity score was 3.1±0.6 (range=1.41–4.12). For each point increase in fidelity, clinics had a significantly reduced likelihood of having a suicide incident (adjusted odds ratio=0.31, 95% confidence interval=0.14–0.69). Exploratory analysis identified significant differences for seven of 17 ZS organizational practices, with the largest effect sizes for suicide-specific quality improvement policies and activities (η2=0.097) and lethal means reduction (η2=0.073).
Conclusions:
These findings support an association between clinics’ use of ZS organizational best practices and lower suicidal behaviors of patients under their care. Findings also support the validity of the ZS Organizational Self-Study instrument.
HIGHLIGHTS
•
A cross-sectional study of 110 mental health clinics found that clinics with higher fidelity to Zero Suicide (ZS) organizational best practices were less likely to have a suicide incident among patients.
•
For each point increase in fidelity to ZS organizational best practices, clinics had a significantly reduced likelihood of having a suicide event.
•
Higher fidelity to seven organizational best practices was significantly associated with a history of no suicide incidents, with the largest effect sizes for suicide-specific clinic quality improvement activities and reduction of lethal means.
•
The ZS Organizational Self-Study instrument had strong psychometric properties.
Suicide is a growing public health crisis. Since 1999, the national prevalence of death by suicide has steadily increased (1). In 2017, it was the 10th leading cause of death in the United States for all age groups and the second for ages 10–44 years (2). Most individuals (83%) who died by suicide accessed general medical or mental health care in the year before their death (3, 4), and suicide and intentional self-harm were the fastest-growing reasons for psychiatric emergency room visits between 2010 and 2014 compared with all other mental health- or substance use–related reasons (5). These facts suggest that health care systems could reduce suicide by improving identification and treatment of individuals at increased risk.
Encouraging health care systems to provide safer, more effective care of suicidal patients has become a national priority (6, 7). In 2012, the Office of the Surgeon General and the National Action Alliance for Suicide Prevention (NAASP), a public-private partnership dedicated to reducing suicide, released a broad national suicide prevention strategy (6). The NAASP Clinical Care and Intervention Taskforce focused on recommendations that were targeted specifically to health care settings (8) and based on an environmental scan of large-scale suicide prevention efforts, including the U.S. Air Force Suicide Prevention Program (9), the quality improvement initiative of the Henry Ford Health System’s Perfect Depression Care (10, 11), Central Arizona Programmatic Suicide Deterrent System, the National Suicide Prevention Lifeline, and others (12). These programs provided compelling evidence that health care systems can reduce suicide through a bundled set of interventions coupled with intentional and sustained leadership and continuous quality improvement activities. The taskforce named this approach Zero Suicide (ZS) to reflect an aspirational goal of preventing all suicides by patients in health systems and to provide a framework for suicide prevention within health care settings. A similar national effort targeting health services in the United Kingdom reduced suicide deaths and, importantly, found that mental health services with a more comprehensive approach had greater reductions, underscoring the need for developing and implementing a systematic model (13). Early adopters of the ZS framework, including Centerstone, a behavioral health system in Tennessee, and the Institute for Family Health, a large federally qualified health center primary care network in New York, observed reductions in suicide incidents of >50% within 3 years (12). The ZS Toolkit was informed by these early adopters and was developed by the Suicide Prevention Resource Center to support large-scale implementation (12, 14), and randomized controlled trials of ZS are under way (15–17).
Hogan and Goldstein Grumet (12) have defined seven core components of the ZS model to improve the identification and treatment of individuals most at risk for suicide. Three of these components address administrative best practices in managing change, including leadership, training, and measuring outcomes and conducting quality improvement. Four additional components reflect established best practices in suicide care, namely, suicide screening and risk assessment (18–21); use of systematic suicide care protocols that include safety planning and lethal means reduction (22–29); evidence-based treatment to address suicidal thoughts and behaviors directly, in addition to other mental health issues (30–34); and provision of support during care transitions, with follow-up after discharge from acute care settings such as “caring contacts” (35–37).
Further highlighting the growing national consensus around these core suicide safer care practices, two national accrediting organizations, The Joint Commission and the Commission on Accreditation of Rehabilitation Facilities, have recently amended their accreditation standards to include suicide screening, risk assessment, and follow-up care after discharge, listing ZS among suggested resources (38, 39). The ZS Toolkit provides tools to assist providers and health care systems in implementing model practices, including the ZS Organizational Self-Study, which assesses fidelity to ZS organizational best practices (40).
Growing support for the model has led to implementation projects across diverse health care systems, states, and tribal nations in the United States (16, 17). Understanding the relationship between fidelity to ZS organizational practices and suicide-related outcomes is critical to support ongoing ZS implementation and research efforts (41–43). The current study examined the relationship between fidelity to the organizational best practices promoted by the ZS model and suicidal behaviors in the year preceding a statewide implementation of ZS in mental health clinics. We expected that clinics with higher fidelity would be less likely to have had patients with suicidal behaviors during the previous year. In addition, we examined the ZS Organizational Self-Study tool’s psychometric properties to support the large-scale implementation project and to inform future use of this instrument.
Methods
Setting
The study focused on community-based mental health clinics not affiliated with hospitals in New York State (NYS) in the year before large-scale ZS implementation. The Suicide Prevention Continuous Quality Improvement (SP-CQI) project was launched in 2017 to support ZS model implementation in outpatient mental health clinics and was a collaborative effort between the NYS Office of Mental Health (NYSOMH) Bureau of Evidence-Based Services and Implementation Science and Psychiatric Services and Clinical Knowledge Enhancement System (PSYCKES) (44), the NYSOMH Suicide Prevention Office (45), and the Center for Practice Innovations at the Columbia University Department of Psychiatry, NYS Psychiatric Institute (NYSPI) (15, 46). The institutional review boards of the Nathan Kline Institute for Psychiatric Research at the NYS Office of Mental Health and NYSPI determined that the study project did not meet the definition of human subjects research.
Sample
All non–hospital-affiliated, community-based, and NYS-licensed mental health clinics (N=321) were invited to participate in the SP-CQI project. Moreover, state-run psychiatric hospital clinics were required to participate but were excluded from this study because of concerns about bias (due to a centralized governing structure mandating participation) and generalizability (NYS is one of the few states to operate a large-scale network of psychiatric hospitals). The study sample included clinics voluntarily enrolled as of April 1, 2017 (N=134 clinics, a 42% recruitment rate), that completed their baseline ZS Organizational Self-Study (N=131, 98%), and were open 6 months before and after the observation period. The final sample included 110 clinics. In preparation for the study, we compared the sizes (number of patients in a sample week) of participating and nonparticipating clinics and client characteristics, including age, race, Medicaid insurance, serious mental illness, and comorbid substance use. Chi-square analysis using the 2015 Patient Characteristics Survey (PCS) data, a biennial NYSOMH survey of mental health programs, detected no statistically significant differences.
Measures and Data Sources
Clinic and client characteristics.
All clinic and client characteristics were extracted from the 2017 PCS (47), including clinic size (number of patients served during the PCS sample week), and clinic type (whether >50% of population served at the clinic served were children).
Fidelity to ZS organizational practices.
Fidelity to ZS organizational practices (ZS fidelity) was assessed before the SP-CQI project implementation with the ZS Organizational Self-Study instrument (40), which was administered to point persons of clinical projects (clinic leadership or quality improvement staff) via SurveyMonkey between February and May 2017 to inform project planning. The tool includes 17 quantitative items (rated on a 5-point scale, with 5 indicating the highest fidelity) and six descriptive questions (excluded from this analysis). Adaptations to the instrument reflected commonly used language in NYS’s clinics and project requirements (see online supplement to this article). The 17 quantitative items were averaged to compute a total ZS fidelity score for each clinic (48, 49).
Suicidal behaviors in the previous year.
Data on suicidal behaviors—specifically, suicide attempts and deaths—were extracted from a state administrative database, the NYS Incident Management Reporting System (NIMRS) (50). NIMRS is a mandatory reporting system for all adverse incidents (e.g., medication reactions and violence) for NYSOMH-licensed programs. All suicide attempts and deaths are required to be reported within 24 hours of discovery. Because the count of suicidal behaviors was highly skewed, study clinics were dichotomized into two groups: no patients with suicidal behavior incidents (N=40) or one or more patients with suicidal behaviors (N=70). In the year of observation (April 2016–2017), most clinics (N=70, 64%) had reported at least one suicide attempt or death incident among their patients (mean±SD=3.1±4.9, median=1, range=0–27), with 22 clinics (20%) reporting one or more deaths (0.25±0.55, range=0–3).
Data Analysis
Statistical analyses were conducted with SAS, version 9.4, and SPSS, version 25; all statistical tests were two-tailed.
Relationship between ZS fidelity and suicidal behaviors among clinic patients.
A logistic regression was conducted to test whether clinic ZS fidelity was associated with patients’ suicidal behaviors in the previous year. We adjusted the model for clinic size by using 2017 PCS data on the number of patients served in the clinic during a sample week (larger clinics are more likely to have a suicide incident) and by clinic type (mostly child services vs. adult services) because of differences in the prevalence of adolescent versus adult suicide attempts (51). Exploratory analyses with analysis of covariance examined differences in fidelity for individual ZS practices among clinics with and without a suicide incident in the previous year and controlled for clinic size but not clinic type (because of parsimony considerations and lack of a finding in hypothesis testing).
Psychometric properties of the ZS fidelity assessment.
We examined the psychometric properties of the ZS Organizational Self-Study instrument because no previously published findings exist. The ZS Organizational Self-Study had high internal consistency (α=0.90). A principal component analysis (PCA) was conducted by using varimax rotation and the Kaiser criterion to identify whether the instrument had any meaningful subscales. Suitability for PCA was tested with the Kaiser-Meyer-Olkin (KMO) test for sampling adequacy, Bartlett’s test of sphericity, and a correlation matrix review. The study data were suitable for PCA; all 17 items were correlated with at least one other item in the scale (r>0.37). The KMO measure was 0.85, with individual item measures ranging from 0.60 to 0.93, and Bartlett’s test of sphericity was statistically significant (p<0.001). The item with the lowest KMO also had the lowest mean score and assessed the inclusion of suicide attempt survivors in clinic policy (item 3). Excluding this item did not change the overall findings. PCA identified four principal components (with eigenvalues >1), which together accounted for 61.4% of the total variance (26.0%, 14.0%, 11.3%, and 10.1%,). However, the grouping of items was inconsistent (e.g., related practices were not grouped together with frequent cross-loading); therefore, the interpretability criterion was not met. This lack of interpretability suggested that the ZS fidelity instrument is best interpreted as a single scale (see online supplement ).
Results
Characteristics of the study clinics and their patients are presented in Table 1. Most clinics were located in urban settings (70%), and 80% predominantly served adults. The participating clinics served 30,257 patients during a typical week.
| Clinics or patients | ||
|---|---|---|
| Characteristic | Nb | % |
| Clinic | ||
| Most patients are children (>50% served are ≤17 years)c | 22 | 20 |
| Urban (≥50,000 people) | 77 | 70 |
| Rural (<50,000 people) | 33 | 30 |
| Federally qualified health center | 8 | 7 |
| One or more suicide-related incident in the previous yeard | 70 | 64 |
| Patients served per week per clinic (median)e | 233 | |
| Patient | M±SD | Range |
|---|---|---|
| % of patients served by clinic in a typically week | ||
| Children (≤17 years) | 30±30 | 0–98 |
| Older adults (≥65 years) | 7±10 | 0–83 |
| Racial-ethnic minority groups | 44±26 | 0–97 |
| Non–English-speaking preference | 10±14 | 0–66 |
| Veteran | 2±2 | 0–9 |
| Medicaid recipient | 77±15 | 25–100 |
| Serious mental illness or serious emotional disturbance | 90±9 | 58–100 |
| Alcohol or substance use disorder | 13±14 | 0–70 |
| Intellectual disability | 12±6 | 0–34 |
| Competitive and integrated employment | <1±<1 | 0–1 |
| Criminal or juvenile justice status | .1±.1 | 0–1 |
| Past-year clinic suicide incidentsd | ||
| All suicidal behaviors | 3.11±4.94 | 0–27 |
| Suicide attempts | 2.88±4.64 | 0–25 |
| Suicide deaths | .25±.55 | 0–3 |
a
Data on patients served were from the biennial Patient Characteristics Survey (PCS) of all New York State Office of Mental Health–licensed programs during a single week of observation.
b
Data are shown for 110 clinics, except for non–English-speaking preference (N=96), veteran (N=105), Medicaid recipient (N=107), serious mental illness or serious emotional disturbance (N=108), competitive and integrated employment (N=105), and criminal or juvenile justice status (N=99).
c
Number of patients served during PCS assessment week and majority adult- or child-serving clinic based on PCS reporting in which the two clinic characteristics were adjusted for in logistic regression analysis.
d
Data on suicide-related incidents (including attempts and deaths) were obtained from the New York State Incident Management Reporting System, a mandatory reporting system for all adverse incidents for New York State Office of Mental Health–licensed mental health programs.
e
M±SD=275.1±226.6, range=10–1,314.
ZS Fidelity for All Clinics in the Year Before Large-Scale ZS Implementation
Clinics’ ZS fidelity assessed with the ZS Organizational Self-Study instrument ranged from very low to high (1.4–4.1 of 5.0), with a mean±SD of 3.1±0.6, reflecting moderate fidelity. Examining individual organizational practice items, we found that the highest rated item was safety planning (item 12), with a mean of 4.1±0.9, followed by screening using a validated instrument (item 8, 4.0±1.3) and routine suicide screening (item 7, mean of 3.9±0.8) (Table 2). The lowest rated items were policy input from suicide attempt survivors (item 3, 1.2±0.6), assessment of staff suicide care confidence and skills (item 4, 1.8±0.9), identifying and measuring suicide death rates (item 19, 2.5±0.9), and lethal means reduction (item 13, 2.6±1.2). Similarly, a high proportion of clinics reported high fidelity (i.e., scored 4 or 5) on safety planning (78%), but few reported high fidelity to lethal means reduction (16%).
| Item | All clinics (N=110) | No suicide event (N=40) | Suicide event (N=70) | ||||||
|---|---|---|---|---|---|---|---|---|---|
| no.a | Item | M | SD | Mb | SD | Mb | SD | Fc | η2 |
| 1 | Leadership commitment to suicide-specific policies | 3.4 | .9 | 3.6 | .9 | 3.3 | .9 | 5.70* | .051 |
| 2 | Leadership commitment to dedicated staffing | 2.7 | 1.0 | 2.6 | 1.1 | 2.7 | 1.0 | .53 | .004 |
| 3 | Survivors have input into clinic policy | 1.2 | .6 | 1.2 | .5 | 1.2 | .6 | .61 | .006 |
| 4 | Staff assessment | 1.8 | .9 | 2.1 | .9 | 1.6 | .8 | 4.23* | .038 |
| 5 | Staff training | 3.2 | 1.3 | 3.5 | 1.4 | 3.0 | 1.2 | 2.96 | .027 |
| 7 | Suicide screening protocol | 3.9 | .8 | 4.1 | .7 | 3.9 | .8 | 3.58 | .032 |
| 8 | Use of validated screening tool | 4.0 | 1.3 | 4.1 | 1.2 | 3.9 | 1.3 | 3.23 | .029 |
| 10 | Suicide risk assessment | 3.4 | 1.1 | 3.7 | 1.1 | 3.3 | 1.1 | 5.28* | .047 |
| 11 | Suicide care pathway for patients at risk | 3.2 | 1.0 | 3.4 | 1.1 | 3.2 | 1.0 | 1.49 | .014 |
| 12 | Safety planning | 4.1 | .9 | 4.3 | 1.0 | 4.0 | .9 | 2.88 | .026 |
| 13 | Lethal means reduction | 2.6 | 1.2 | 3.0 | 1.3 | 2.3 | 1.1 | 8.38** | .073 |
| 14 | Suicide-specific treatment | 2.9 | .8 | 3.0 | 1.0 | 2.9 | .8 | .90 | .008 |
| 16 | Outreach after missed appointments | 3.4 | 1.0 | 3.6 | 1.0 | 3.2 | 1.0 | 4.18* | .038 |
| 17 | Acute care transition support | 3.7 | .9 | 3.9 | .9 | 3.6 | .9 | 4.78* | .043 |
| 18 | Reviewing suicide deaths | 3.0 | 1.2 | 3.3 | 1.4 | 2.8 | 1.1 | 6.17 | .055 |
| 19 | Measuring suicide deaths | 2.5 | .9 | 2.7 | 0.9 | 2.5 | .9 | 2.13 | .019 |
| 20 | Suicide-specific quality improvement activities | 3.0 | 1.0 | 3.3 | 1.1 | 2.8 | .9 | 11.48** | .097 |
| Total ZS fidelity scored | 3.1 | .6 | 3.2 | .7 | 2.9 | .6 | 9.44** | .080 | |
a
Item number in the ZS Organizational Self-Study adapted from the New York State Office of Mental Health Suicide Prevention Continuous Quality Improvement project. This instrument is a 23-item survey with 17 quantitative items; its six qualitative items (6, 9, 15, and 21–23) were excluded from the analysis. Items were rated on a 5-point Likert scale, where 1 indicates the lowest and 5 the highest fidelity.
b
Unadjusted means are presented for clinics with and without a suicide event in the previous year.
c
Clinics with and without a suicide event were compared by using analyses of covariance controlled for clinic size (number of patients seen during the sample week of the New York State Office of Mental Health 2017 Patient Characteristics Survey). df=1, 107.
d
The 17 quantitative items were averaged for a total ZS fidelity score.
**
p<0.01, *p<0.05 for mean difference between groups.
Association Between ZS Fidelity and Suicidal Behaviors Among Patients
Results of the logistic regression model are presented in Table 3. After adjusting for patient census and population type served, we found that clinics with higher ZS fidelity had 0.31 lower odds of having a client with suicidal behavior during the previous year (adjusted odds ratio=0.31, 95% confidence interval=0.14–0.69). In other words, for each unit increase (i.e., one point on a 1–5 scale) on the ZS fidelity scale, clinics were significantly less likely to have any patients with suicidal behaviors. The model explained 33% (Nagelkerke R2) of the variance in suicidal behaviors.
| Clinic characteristic | β | SE | AORb | 95% CI | p |
|---|---|---|---|---|---|
| Zero Suicide fidelityc | −.1.16 | .40 | .31 | .14–.69 | <.01 |
| Clinic sized | .007 | .002 | 1.01 | 1.00–1.01 | <.001 |
| Clinic type (reference: adult-serving clinic)e | −.42 | .58 | .49 | .21–2.06 | .472 |
a
Full model results χ2=30.06, df=3, p<0.001.
b
We adjusted the model for clinic size (median of 233 patients served per week) because larger clinics were more likely to have a suicide incident. We also adjusted the model for clinic type because of differences in the prevalence of adolescent versus adult suicide attempts. No other variables were included.
c
Fidelity was measured on a scale from 1 to 5. For each 1-point increase on the Zero Suicide fidelity scale, clinics were significantly less likely to have any patients with suicidal behaviors.
d
Clinic size was defined as the number of patients served in the sample week as assessed in the New York City Office of Mental Health 2017 Patient Characteristics Survey (PCS).
e
On the basis of the PCS, clinics were categorized as serving predominantly adults (>50% served were ≥18 years old) or children (>50% patients served are ≤17 years old).
Differences in ZS Organizational Practices Between Clinics With and Without Suicidal Incidents
A statistically significant difference between clinics with and without a suicide incident was observed for total average fidelity scores and for seven of the 17 organizational practice items (Table 2). Medium effect sizes were observed for two items: quality improvement activities focused on suicide prevention (item 20) and lethal means reduction (item 13) (η2=0.097 and 0.073, respectively). Examination of the proportion of clinics who achieved high fidelity (i.e., scored 4 or 5 on the ZS Organizational Self-Study instrument) on these two items identified marked differences for clinics with and without a suicide incident. Nearly half (45%, N=18) of clinics without a suicide incident reported that they met the quality improvement criteria compared with fewer than a quarter (23%, N=16) of clinics with an incident in the previous year. Small but statistically significant effects were observed for five other ZS fidelity organizational practice items: leadership commitment (item 1), assessments of confidence in suicide care and of skills among staff (item 4), suicide risk assessments (item 10), engaging hard-to-reach and no-show patients (item 16), and following up with patients who have been discharged from acute settings (item 17).
Discussion
To our knowledge, this is the first study that reports an association between greater fidelity to ZS organizational practices and lower risk for suicidal behaviors. Specifically, after adjusting for patient census and population type served (adult vs. child), we found that the results supported the hypothesis that clinics with higher fidelity to the organizational practices promoted by the ZS model were less likely to have suicidal attempts or deaths among their patients. This finding was observed before a large-scale ZS implementation and extends preliminary research suggesting that the comprehensive approach encapsulated within the ZS framework is associated with fewer suicidal behaviors among those under care (10, 11). In addition, our findings suggest that specific ZS practices may be important priorities for suicide prevention efforts, particularly initiating suicide-focused quality improvement processes and reducing lethal means.
The study results indicate that the ZS Organizational Self-Study instrument has high internal consistency and concurrent validity. Specifically, the instrument could distinguish between clinics with and without suicide incidents. The instrument is brief, accessible, and in the public domain, and it does not require special training or expert raters (40). Self-assessment is important because it allows clinical programs to use the instrument in order to support internal quality improvement processes and offers a feasible approach to examining fidelity in large-scale implementation initiatives (52, 53).
As described above, the ZS framework was based on innovative quality improvement projects that identified death by suicide as a problem within health care systems and leveraged leadership commitment to monitor and reduce such deaths among individuals under care (9, 11, 54). It is noteworthy that among the 17 ZS organizational practices, the quality improvement infrastructure item had the highest effect size. Organizational best practices for suicide-specific quality improvement were defined as having suicide care embedded in the medical chart, written clinical workflows for suicide care, and data collection and review by clinical teams (e.g., data on the quality of patient suicide care plans). Nearly half (45%) of clinics without a suicide incident reported that they met the quality improvement criteria compared with fewer than one-quarter (23%) of clinics with an incident in the year before. These findings underscore the importance of developing a sustainable data-monitoring and quality improvement infrastructure to support suicide prevention efforts.
In exploratory analyses, lethal means reduction also emerged as an important ZS practice, with the largest mean difference between clinics with and without a suicide incident. Achieving high fidelity to the lethal-means-reduction item requires documentation in safety plans as a standard practice, in addition to policies addressing clinician training, family inclusion in means reduction, and confirmation of means reduction. Interestingly, in this sample, most clinics (78%) reported high fidelity to safety planning, but few (16%) reported high fidelity to lethal means reduction, even though most safety planning interventions are supposed to include lethal means reduction. These findings suggest that staff may require additional training in lethal means reduction and safety planning to be comfortable and effective in these integrated practices. Safety planning interventions that incorporate lethal means reduction are associated with a 45% decrease in suicidal behaviors over 6 months (27). Moreover, clinic policies clarifying expectations for patients, their families, and staff to implement and confirm means reduction may be required to maximize the benefits of safety planning and means reduction counseling. These findings align with literature highlighting the role of lethal means reduction in reducing suicide (22, 23, 55). Future research may examine whether large-scale interventions, such as the SP-CQI initiative, can increase fidelity to lethal means reduction and other best practices and can decrease suicidal behaviors.
This study had several strengths and limitations. Its strengths included data from 110 mental health clinics serving a large and diverse population and the use of state-mandated reporting data as an objective measure for suicidal behaviors. Limitations included the following. First, our findings may not generalize to other treatment settings or patient populations. Second, we did not differentiate between suicide attempts and deaths. Third, the ZS Organizational Self-Study is a self-reported instrument, which may introduce bias. The role of self-reported fidelity has been debated in the literature but can offer a reliable, valid, and cost-effective method in specific contexts (41, 52, 53, 56, 57). Moreover, NIMRS, the resource we used for data indicating suicidal behaviors, is a state administrative database for monitoring serious incidents and adverse incidents and was not designed for research purposes. We could not include unreported suicidal behaviors, such as incidents of which the agency was unaware or suicidal behaviors that did not meet reporting criteria. In addition, although the size of the data set allowed for adjustment for clinic size and type, it did not have the statistical power to enable adjustment for other clinic- and patient-level characteristics that may affect organizational practices and outcomes, an important area for future study.
This cross-sectional study examined suicidal behaviors in the year before the fidelity assessment; the optimal period of such observations is unclear, given the evolution of organizational practices over time. Longitudinal study is needed to investigate fidelity over time and the relationship between gains in fidelity to changes in suicide outcomes. Finally, the exploratory analyses did not account for multiple comparisons, increasing the chance of type I errors, and ceiling effects for select organizational practice items (e.g., safety planning) may have introduced type II errors.
Conclusions
The findings of this study suggest that high fidelity to ZS organizational best practices in outpatient mental health clinics may reduce suicidal behaviors among patients. Exploratory analyses suggested that clinic engagement in suicide-specific quality improvement activities and in strategic development of effective policy- and protocol-based lethal means reduction may be particularly important for reducing suicide risk. Our findings also indicate that the ZS Organizational Self-Study instrument has high internal consistency and concurrent validity with patients’ suicidal behaviors, suggesting it is a useful tool for health care systems.
Footnote
Dr. Stanley has received royalties from the Research Foundation for Mental Hygiene for the commercial use of the Columbia–Suicide Severity Rating Scale. The other authors report no financial relationships with commercial interests.
Supplementary Material
File (appi.ps.202000525.ds001.pdf)
- View/Download
- 398.87 KB
References
1.
Curtin SC, Warner M, Hedegaard H: Increase in Suicide in the United States, 1999–2014. Atlanta, US Centers for Disease Control and Prevention, 2016. https://www.cdc.gov/nchs/products/databriefs/db241.htm. Accessed January 28, 2021
2.
WISQARS (Web-Based Injury Statistics Query and Reporting System). Atlanta, US Centers for Disease Control and Prevention, 2018. www.cdc.gov/injury/wisqars. Accessed June 21, 2018
3.
Ahmedani BK, Simon GE, Stewart C, et al: Health care contacts in the year before suicide death. J Gen Intern Med 2014; 29:870–877
4.
Fontanella CA, Warner LA, Hiance-Steelesmith DL, et al: Service use in the month and year prior to suicide among adults enrolled in Ohio Medicaid. Psychiatr Serv 2017; 68:674–680
5.
Hooker EA, Mallow PJ, Oglesby MM: Characteristics and trends of emergency department visits in the United States (2010–2014). J Emerg Med 2019; 56:344–351
6.
2012 National Strategy for Suicide Prevention: Goals and Objectives for Action. Washington, DC, US Department of Health and Human Services (HHS), Office of the Surgeon General, and National Action Alliance for Suicide Prevention, 2012. https://www.adolescentwellness.org/wp-content/uploads/2012/11/2012-National-Strategy-For-Suicide-Prevention.pdf
7.
Project 2025. New York, American Foundation for Suicide Prevention, 2019. https://project2025.afsp.org. Accessed January 28, 2021
8.
Suicide Care in Systems Framework. Washington, DC, National Action Alliance for Suicide Prevention, 2011. https://theactionalliance.org/sites/default/files/clinicalcareinterventionreport.pdf
9.
Knox KL, Pflanz S, Talcott GW, et al: The US Air Force suicide prevention program: implications for public health policy. Am J Public Health 2010; 100:2457–2463
10.
Coffey CE: Building a system of perfect depression care in behavioral health. Jt Comm J Qual Patient Saf 2007; 33:193–199
11.
Hampton T: Depression care effort brings dramatic drop in large HMO population’s suicide rate. JAMA 2010; 303:1903–1905
12.
Hogan MF, Goldstein Grumet J: Suicide prevention: an emerging priority for health care. Health Aff 2016; 35:1084–1090
13.
While D, Bickley H, Roscoe A, et al: Implementation of mental health service recommendations in England and Wales and suicide rates, 1997-2006: a cross-sectional and before-and-after observational study. Lancet 2012; 379:1005–1012
14.
Zero suicide toolkit. New York, Zero Suicide Institute, 2019. https://zerosuicide.sprc.org/toolkit. Accessed March 20, 2017
15.
Labouliere CD, Vasan P, Kramer A, et al: “Zero Suicide”—A model for reducing suicide in United States behavioral healthcare. Suicidologi 2018; 23:22–30
16.
SM-17-006 Individual Grant Awards. Bethesda, MD, Substance Abuse and Mental Health Services Administration, 2018. https://www.samhsa.gov/grants/awards/2018/SM-17-006. Accessed July 9, 2018
17.
NIMH Funds 3 ‘Zero Suicide’ Grants. Bethesda, MD, National Institute of Mental Health, 2016. https://www.nimh.nih.gov/news/science-news/2016/nimh-funds-3-zero-suicide-grants.shtml. Accessed July 23, 2018
18.
Posner K, Brown GK, Stanley B, et al: The Columbia–Suicide Severity Rating Scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatry 2011; 168:1266–1277
19.
Simon GE, Rutter CM, Peterson D, et al: Does response on the PHQ-9 Depression Questionnaire predict subsequent suicide attempt or suicide death? Psychiatr Serv 2013; 64:1195–1202
20.
Roaten K, Johnson C, Genzel R, et al: Development and implementation of a universal suicide risk screening program in a safety-net hospital system. Jt Comm J Qual Patient Saf 2018; 44:4–11
21.
Fowler JC: Suicide risk assessment in clinical practice: pragmatic guidelines for imperfect assessments. Psychotherapy 2012; 49:81–90
22.
Houtsma C, Butterworth SE, Anestis MD: Firearm suicide: pathways to risk and methods of prevention. Curr Opin Psychol 2018; 22:7–11
23.
Miller M, Hemenway D: Guns and suicide in the United States. N Engl J Med 2008; 359:989–991
24.
Conwell Y, Duberstein PR, Connor K, et al: Access to firearms and risk for suicide in middle-aged and older adults. Am J Geriatr Psychiatry 2002; 10:407–416
25.
Stanley B, Brown GK: Safety planning intervention: a brief intervention to mitigate suicide risk. Cogn Behav Pract 19: 256–264, 2012. doi:
26.
Bryan CJ, Mintz J, Clemans TA, et al: Effect of crisis response planning on patient mood and clinician decision making: a clinical trial with suicidal US soldiers. Psychiatr Serv 2018; 69:108–111
27.
Stanley B, Brown GK, Brenner LA, et al: Comparison of the safety planning intervention with follow-up vs usual care of suicidal patients treated in the emergency department. JAMA Psychiatry 2018; 75:894–900
28.
Zonana J, Simberlund J, Christos P: The impact of safety plans in an outpatient clinic. Crisis 2018; 39:304–309
29.
Czyz EK, King CA, Biermann BJ: Motivational interviewing-enhanced safety planning for adolescents at high suicide risk: a pilot randomized controlled trial. J Clin Child Adolesc Psychol 2019; 48:250–262
30.
Jobes DA, Wong SA, Conrad AK, et al: The collaborative assessment and management of suicidality versus treatment as usual: a retrospective study with suicidal outpatients. Suicide Life Threat Behav 2005; 35:483–497
31.
Stanley B, Brown G, Brent DA, et al: Cognitive-behavioral therapy for suicide prevention (CBT-SP): treatment model, feasibility, and acceptability. J Am Acad Child Adolesc Psychiatry 2009; 48:1005–1013
32.
Brown GK, Ten Have T, Henriques GR, et al: Cognitive therapy for the prevention of suicide attempts: a randomized controlled trial. JAMA 2005; 294:563–570
33.
DeCou CR, Comtois KA, Landes SJ: Dialectical behavior therapy is effective for the treatment of suicidal behavior: a meta-analysis. Behav Ther 2019; 50:60–72
34.
Linehan MM, Comtois KA, Ward-Ciesielski EF: Assessing and managing risk with suicidal individuals. Cog Behav Pract 19: 218–232, 2012. doi:
35.
Luxton DD, June JD, Comtois KA: Can postdischarge follow-up contacts prevent suicide and suicidal behavior? A review of the evidence. Crisis 2013; 34:32–41
36.
Currier GW, Brown GK, Brenner LA, et al: Rationale and study protocol for a two-part intervention: safety planning and structured follow-up among veterans at risk for suicide and discharged from the emergency department. Contemp Clin Trials 2015; 43:179–184
37.
Knesper D: Continuity of Care for Suicide Prevention and Research: Suicide Attempts and Suicide Deaths Subsequent to Discharge from The Emergency Department or Psychiatry Inpatient Unit. Newton, MA, American Association of Suicidology, Suicide Prevention Resource Center, 2010. http://www.sprc.org/sites/default/files/migrate/library/continuityofcare.pdf
38.
Suicide Prevention. Tucson, AZ, CARF International, 2019. http://www.carf.org/QPN_SuicidePrevention. Accessed August 12, 2020
39.
Suicide Prevention. Oakbrook Terrace, IL, The Joint Commission. 2019. https://www.jointcommission.org/resources/patient-safety-topics/suicide-prevention. Accessed July 25, 2019
40.
Zero Suicide Organizational Self-Study. New York, Zero Suicide Institute, 2018. https://zerosuicide.edc.org/resources/zero-suicide-organizational-self-study. Accessed June 14, 2018
41.
Proctor E, Silmere H, Raghavan R, et al: Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health Ment Health Serv Res 2011; 38:65–76
42.
Aarons GA, Hurlburt M, Horwitz SMC: Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health Ment Health Serv Res 2011; 38:4–23
43.
Carroll C, Patterson M, Wood S, et al: A conceptual framework for implementation fidelity. Implement Sci 2007; 2:40
44.
About Psychiatric Services and Clinical Knowledge Enhancement System (PSYCKES). Albany, New York Office of Mental Health, 2017. https://www.omh.ny.gov/omhweb/psyckes_medicaid/about. Accessed June 15, 2018
45.
Suicide Prevention: What Is New York State Doing to Prevent Suicide? Albany, New York Office of Mental Health: Office of Suicide Prevention, 2019. https://www.omh.ny.gov/omhweb/suicide_prevention. Accessed July 1, 2019
46.
Center for Practice Innovations. New York City, Columbia University Department of Psychiatry New York State Psychiatric Institute 2019. https://practiceinnovations.org. Accessed July 6, 2018
47.
The Patient Characteristics Survey (PCS). Albany, New York State Office of Mental Health 2018. https://www.omh.ny.gov/omhweb/pcs/submissions. Accessed July 19, 2018
48.
Norman G: Likert scales, levels of measurement and the “laws” of statistics. Adv Health Sci Educ Theory Pract 2010; 15:625–632
49.
DiStefano C, Zhu M, Mîndrilaˇ D: Understanding and using factor scores: considerations for the applied researcher. Pract Assess Res Eval 2009; 14:1–11
50.
New York State Incident Management and Reporting System. Albany, New York Office of Mental Health, 2018. https://omh.ny.gov/omhweb/dqm/bqi/nimrs. Accessed June 11, 2018
51.
Curtin SC, Heron M: Death Rates Due to Suicide and Homicide Among Persons Aged 10–24: United States, 2000–2017. Atlanta, US Centers for Disease Control and Prevention, 2019. https://www.cdc.gov/nchs/products/databriefs/db352.htm. Accessed January 28, 2021
52.
Margolies PJ, Humensky JL, Chiang I-C, et al: Is there a role for fidelity self-assessment in the individual placement and support model of supported employment? Psychiatr Serv 2017; 68:975–978
53.
McGrew JH, White LM, Stull LG: Self-assessed fidelity: proceed with caution: in reply. Psychiatr Serv 2013; 64:394
54.
Coffey MJ, Coffey CE, Ahmedani BK: Suicide in a health maintenance organization population. JAMA Psychiatry 2015; 72:294–296
55.
Mann JJ, Apter A, Bertolote J, et al: Suicide prevention strategies: a systematic review. JAMA 2005; 294:2064–2074
56.
Rollins AL, Kukla M, Salyers MP, et al: Comparing the costs and acceptability of three fidelity assessment methods for assertive community treatment. Adm Policy Ment Health 2017; 44:810–816
57.
Rollins AL, McGrew JH, Kukla M, et al: Comparison of assertive community treatment fidelity assessment methods: reliability and validity. Adm Policy Ment Health 2016; 43:157–167
Information & Authors
Information
Published In
History
Received: 13 July 2020
Revision received: 18 November 2020
Accepted: 21 December 2020
Published online: 18 March 2021
Published in print: October 01, 2021
Keywords
Authors
Funding Information
This work was supported by a New York State Office of Mental Health contract to Dr. Finnerty and by National Institute of Mental Health grant R01-MH-112139 (to Dr. Stanley). The authors thank Katrina Vega, M.S.W., for manuscript support and the Suicide Prevention Continuous Quality Improvement implementation team, Prabu Vasan, L.C.S.W.-R., Anni Cummings, L.M.S.W., Paulina Prejbisz, and Debbie Seecharran.
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).