Suicide Risk Assessment and Prevention: A Systematic Review Focusing on Veterans
Abstract
Objective:
Suicide rates in veteran and military populations in the United States are high. This article reviews studies of the accuracy of methods to identify individuals at increased risk of suicide and the effectiveness and adverse effects of health care interventions relevant to U.S. veteran and military populations in reducing suicide and suicide attempts.
Methods:
Trials, observational studies, and systematic reviews relevant to U.S. veterans and military personnel were identified in searches of MEDLINE, PsycINFO, SocINDEX, and Cochrane databases (January 1, 2008, to September 11, 2015), on Web sites, and in reference lists. Investigators extracted and confirmed data and dual-rated risk of bias for included studies.
Results:
Nineteen studies evaluated accuracy of risk assessment methods, including models using retrospective electronic records data and clinician- or patient-rated instruments. Most methods demonstrated sensitivity ≥80% or area-under-the-curve values ≥.70 in single studies, including two studies based on electronic records of veterans and military personnel, but specificity varied. Suicide rates were reduced in six of eight observational studies of population-level interventions. Only two of ten trials of individual-level psychotherapy reported statistically significant differences between treatment and usual care.
Conclusions:
Risk assessment methods have been shown to be sensitive predictors of suicide and suicide attempts, but the frequency of false positives limits their clinical utility. Research to refine these methods and examine clinical applications is needed. Studies of suicide prevention interventions are inconclusive; trials of population-level interventions and promising therapies are required to support their clinical use.
Despite the large public health impact of suicide in the United States (1), research supporting clinical guidelines for suicide prevention has been limited. Universal evaluation for suicide risk (screening) in general medical practice is currently not recommended (2,3), although a Sentinel Event Alert regarding identification of suicide risk has been issued by The Joint Commission (4). However, primary care clinicians and other health care practitioners are often well positioned to serve as initial contacts in the assessment and referral of high-risk individuals. During the year prior to suicide, an estimated 83% of suicidal individuals used health care services, including 64% who had contact with primary care and 45% who had contact with mental health care clinicians (5).
Efforts to prevent suicide generally include referrals for treatment of underlying conditions, psychotherapy, and management of imminent risk, such as hospitalization and development of safety plans (6,7). In addition to individual-level approaches, an increasing number of suicide prevention initiatives have been implemented at organizational, health system, and community levels. However, their influence on suicide prevention remains unclear because few studies demonstrating effectiveness have been published (7).
Suicide rates in the United States, including rates in veteran and military populations, are high (8,9). Rates of suicide increased among active duty soldiers during the wars in Afghanistan and Iraq (10), and between 2000 and 2010, the suicide rate among veterans surpassed the rate among civilians (11). Female veterans are at especially high risk compared with women who are not veterans (11). These trends have led to new initiatives and additional research funding to address suicide risk detection and prevention in both the U.S. Department of Veterans Affairs (VA) and the Department of Defense. As part of this effort, this systematic review updates evidence of the accuracy of methods to identify individuals at increased risk of suicide and the effectiveness and adverse effects of health care interventions relevant to U.S. veteran and military populations in reducing suicide and suicide attempts. This review adds to existing research (12–16) by focusing on studies applicable to veteran and military populations, including risk assessment studies and trials of interventions, and evaluating the strength of evidence by using established methods.
Methods
Scope, Key Questions, and Analytic Framework
The scope and key questions were determined during a topic refinement process that included a preliminary review of published peer-reviewed literature; discussion with internal partners, policy makers, and investigators; and consultation with content experts and key stakeholders. This review followed an established systematic review methodology (17), and a protocol was posted to the PROSPERO International Prospective Register of Systematic Reviews Web site (http://www.crd.york.ac.uk/PROSPERO; registration number CRD42015019089) (18). A technical report describes additional methods (19). Institutional review board approval was not required for this systematic review.
We created an analytic framework outlining the key questions, patient populations, interventions, and outcomes of the review. [A figure depicting the framework is included in an online supplement to this article.] Key questions addressed the accuracy and adverse effects of methods to identify individuals at increased risk of suicide and attempts and the effectiveness and adverse effects of suicide prevention interventions in reducing rates of suicide and attempts. Interventions included health care services directed toward populations (for example, educational programs and practice changes) and individuals (for example, psychotherapy and case management).
Outcomes of interventions included reduced suicide and other suicidal self-directed violence. Suicidal self-directed violence is behavior against oneself that deliberately results in injury or the potential for injury with evidence of suicidal intent (that is, nonfatal suicide attempts) (20,21). Suicide is a fatal outcome of suicidal self-directed violence. Suicidal ideation and nonsuicidal self-directed violence, or composite outcomes that include these thoughts or behaviors, were outside the scope of this review. Adverse effects of these interventions included health outcomes as well as other outcomes, including patient anxiety and distress, labeling, and stigma.
A research librarian conducted electronic database searches of MEDLINE by using PubMed, PsycINFO, SocINDEX, Cochrane Central Register of Controlled Trials, and the Cochrane Database of Systematic Reviews between January 1, 2008, and September 11, 2015, to capture studies published since previous systematic reviews on this topic (6,7,22). [A list of search terms is provided in the online supplement .] Additional sources included a search of relevant research and organizations’ Web sites and citations from reference lists, reviews, conference proceedings, and experts.
Study Selection
Prespecified eligibility criteria were used to determine inclusion of abstracts and articles [see online supplement ]. English-language full-text articles identified as potentially relevant were independently examined by two of the authors. At least one reviewer for each study was a physician or a doctoral-level investigator experienced in the field; disagreements were resolved through consensus by using a third reviewer.
Consistent with the previous reviews (6,7,22), eligible studies included populations of veterans, military personnel, and demographically comparable nonveteran adults ages 18 and older from the United States, United Kingdom, Canada, New Zealand, and Australia. These countries were chosen because of their similarities to the United States in terms of health care services and their involvement in Operation Iraqi Freedom and Operation Enduring Freedom and conflicts in Iraq and Afghanistan. Studies enrolling participants substantially different from the general veteran population (for example, children and adolescents) or with serious comorbidities (for example, advanced cancer) were outside the scope of this review.
Studies were included that evaluated the diagnostic accuracy or adverse effects of methods to assess risk of suicide and attempts, including instruments, checklists, and other approaches appropriate for clinical settings. Outcomes included sensitivity, specificity, area under the receiver-operator characteristic curve (AUC), and similar measures. Studies were excluded that primarily determined associations between individual risk factors and suicide and attempts, evaluated psychometric characteristics of instruments, or provided only descriptions of methods without reporting measures of accuracy.
Studies of the efficacy or effectiveness of interventions were eligible if the interventions were specifically designed to prevent suicide and attempts, including randomized controlled trials (RCTs), observational studies with comparison groups, and systematic reviews of studies with these study designs. These included primary, secondary, and tertiary prevention intervention studies of health care services clinically relevant to medical practice in the United States. Studies of interventions that were intended to primarily treat coexisting conditions and studies of pharmacotherapy were outside the scope of this review.
Data Extraction and Quality Assessment
Details of the study design, setting, patient population, methodology, and results were abstracted into a customized, prepiloted database by one author and confirmed by a second. Two authors independently assessed the risk of bias of included studies by using prespecified criteria for RCTs (23), observational studies (24,25), and diagnostic-accuracy studies (24,26). Each study was given an overall summary assessment of low, high, or unclear risk of bias. Discrepancies were resolved through consensus by using a third author.
Data Synthesis
Measures, interventions, outcomes, and study participants were too heterogeneous to combine in statistical meta-analysis, and conclusions are based on qualitative synthesis of the findings that emphasizes the strongest studies (that is, best evidence approach). The overall strength of evidence for studies of interventions was determined by using a method developed for the Agency for Healthcare Research and Quality’s Evidence-Based Practice Centers (27). This method does not provide strength-of-evidence grades for diagnostic-accuracy studies.
Results
Results of the literature search and selection process are summarized in a literature flow diagram [see figure in online supplement ]. Of 7,788 citations reviewed, 681 articles were selected for full-text review, and 37 studies met inclusion criteria.
Methods to Identify Suicide Risk
Nineteen studies evaluated the accuracy of methods to identify suicide risk (28–46) [see table in online supplement ]. No studies described the adverse effects of risk assessment methods. Five studies evaluated methods that could be offered to general populations (universal or primary prevention) (32,36,37,39,42); nine studies evaluated methods that target subpopulations likely to be at increased risk of suicide, such as patients presenting to emergency departments (selective or secondary prevention) (28–30,33,34,38,40,41,46); and five studies evaluated methods for individuals who have already been identified as having increased risk, such as patients admitted to a hospital for suicidal ideation (tertiary prevention) (31,35,43–45). Studies enrolled participants from the community, online, emergency departments, and psychiatry services or used data from existing patient medical records or administrative databases. Six studies included veterans or military personnel specifically (28–31,36,37). Most studies included high-risk populations, such as adults presenting to emergency departments with suicide attempts (33,38,40,43–45) or who had psychiatric hospitalizations or psychiatric risk factors that indicated higher risk of suicide (28–31,34–36,39,41,46), most commonly depression or previous suicide attempts.
Thirteen studies used one or more clinician-rated or patient–self-reported instruments to assess individual levels of risk (28–30,32–35,38,42–46). Instruments incorporated questions and scales indicating the presence and severity of known or suspected suicide risk factors, and the studies determined how well participants’ responses predicted suicidal behaviors. In one study, participants’ responses to nine separate scales were analyzed in various combinations to determine a model with the best diagnostic accuracy (34).
Six studies used existing data from electronic medical records or administrative databases to identify suicide risk factors in populations of patients and then stratified the patient populations by levels of suicide risk by using regression analysis (31,36,37,39–41). Of these studies, three derived models from development data sets and tested them in validation data sets (37,40,41).
Six of the 19 studies had important methodological limitations resulting in high risk-of-bias ratings (28,29,35,43–45); risk of bias was unclear in ten studies (30–34,38,39,41,42,46), and low in three studies (36,37,40). Major limitations of studies with high or unclear risk of bias included small sample sizes, including six studies with sample sizes less than 200 (28,35,38,43–45), high or unclear loss to follow-up (28,29,33–35,37,38,44,45), and potentially biased participant selection (28,30–32,34,35,38,43–46).
Results of studies indicated estimates of sensitivity ranging from 50% to 100% (with one outlier at 11%) and AUC values from .57 to .97 (Figures 1 and 2) [also see tables in the online supplement ]. The majority of methods had estimates of sensitivity ≥80% or AUC ≥ .70, a threshold suggesting fair or better discrimination between patients with and without suicide or suicide attempts. These methods include SAD PERSONS and variations (33), Suicide Opinion Questionnaire (35), ReACT Self-Harm Rule (40), Suicidal Ideation Attributes Scale (42), modification of the Affective Intensity Rating Scale (43), Suicide Trigger Scale (44,45), Schedule for Nonadaptive and Adaptive Personality self-harm subscale (46), suicide potential index subscale of the Personality Assessment Inventory (28), a decision tree with predictors of suicide attempts (31), and prediction models incorporating electronic database records (34,36,37,41).
FIGURE 1. Sensitivity and specificity estimates reported in 14 studies of suicide risk assessment methodsa
aFor studies reporting multiple results based on different cutpoints, results for the method with the highest sensitivity are shown [see online supplement for a description of the included studies].
FIGURE 2. AUC values reported in 12 studies of suicide risk assessment methodsa
aAUC, area under the receiver-operator characteristic curve. For studies reporting multiple results, results for the method with the highest AUC are shown. [For the order of the studies, see table in the online supplement .]
All three studies with low risk of bias evaluated methods derived from large patient databases (36,37,40). The ReACT Self-Harm Rule is a four-item model developed from data from 18,680 patients in England presenting to emergency departments with self-harm. Items include self-harm in the past year, living alone or homelessness, cutting as a method of harm, and treatment for a current psychiatric disorder. The ReACT Self-Harm Rule demonstrated sensitivity of 88% and specificity of 24% for predicting suicide within six months (40).
A study using data from nearly six million VA patients to create a prediction model to stratify patients according to their risk of suicide within the next year reported an AUC of .761 (95% confidence interval [CI]=.751–.771) (37). Results suggested that the model was a better predictor of suicide than the current clinical practice standard of health care providers “flagging” medical records of individuals believed to be at higher risk of suicide.
The only study of military personnel was based on the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS) and included 40,820 active duty U.S. Army soldiers hospitalized with psychiatric diagnoses (36). A risk algorithm to predict suicides within one year of hospitalization was developed from administrative data systems and demonstrated AUCs as high as .89.
Health Care Service Interventions for Suicide Prevention
Interventions directed toward populations.
Eight studies of the effectiveness of population-level interventions met inclusion criteria (Table 1) (47–54). One study evaluated a secondary prevention intervention (51), and the others targeted primary prevention. These initiatives included components generally categorized as education, awareness, individual health, and individual risk monitoring. One study was designed as a retrospective cohort study (48), six were before-after studies (47,49,51–54), and one was a postintervention series (50) for which risk-of-bias criteria were not applicable.
| Study | Design | Population | Intervention | Results | Risk-of-bias rating |
|---|---|---|---|---|---|
| Coffey, 2007 (47) | Before-after | General mental health and substance abuse patients in a U.S. health care system; baseline year 2000, start-up year 2001, follow-up 2002–2005 | Henry Ford Health System’s Perfect Depression Care initiative utilized 6 aims and 10 rules from the Institute of Medicine Crossing the Quality Chasm report and focused on improving partnership with patients, clinical care (planned care model), access, and information flow. | Suicide rate (per 100,000): 89 baseline, 77 start-up year, 22 follow-up average (p=.007 compared with baseline) | NAb |
| Harris, et al., 2008 (48) | Retrospective cohort | 7,760 patients ages 15–29, 59.7% male, with psychotic disorders receiving mental health services in Victoria, Australia, 1991–1999 | The Early Psychosis Program involved inpatient specialized care at the Early Psychosis Prevention and Intervention Center for up to 24 months. All patients receiving care in the program were compared with patients receiving nonspecialist adult mental health care from other clinics in the area. | Suicide rate across 8.5 years: 3.8% intervention versus 4.2% usual care (p=.84) | Unclear |
| Joffe, 2008 (51) | Before-after | University of Illinois students and citizens of Champaign County, 1984–2005 | Secondary prevention intervention that mandated students with suicide attempts or threats to receive 4 treatment sessions conducted by mental health professionals. Failure to comply resulted in sanctions. | Suicide rate (per 100,000): 6.91 preintervention versus 3.78 postintervention after 21 years of the program (p<.05) | NAb |
| Knox et al., 2010 (49); Knox et al., 2003 (55) | Before-after | >5 million service personnel in the U.S. Air Force, 1981–2008 | An 11-component initiative was implemented starting in 1997: leadership involvement, suicide prevention education, commander guidelines for use of mental health services, community prevention services, community education and training, investigative interview policy, trauma stress response, integrated delivery system and community action information board, limited-privilege suicide prevention program (increased confidentiality), assessment, and suicide event surveillance. | Mean quarterly suicide rate (per 100,000): 3.033 preintervention versus 2.387 postintervention (p<.01); relative risk of suicide pre- versus postimplementation: .67 (95% CI=.57–.80) | NAb |
| Mishara and Martin, 2012 (52) | Before-after | 4,178 members of the Montreal police in Quebec, Canada, 1997–2008 | The Together for Life Suicide Prevention Program consisted of education, police resources, training for supervisors and union representatives, and a publicity campaign. | Suicide rate (per 100,000): 30.5 preintervention versus 6.4 postintervention (p=.008) | NAb |
| Walrath et al., 2015 (53) | Ecological comparison | Youths and adults in 479 counties across the United States, 2007–2010 | The Garrett Lee Smith youth suicide prevention program consisted of education, gatekeeper training, screening activities, improvement of linkages to services, crisis hotlines, and community partnerships. The study compared counties that implemented gatekeeper training with matched counties without training. | At 1 and 2 years after training, no differences for adults age ≥19 | NAb |
| Warner et al., 2011 (50) | Postintervention series | 40,283 soldiers in a deployed U.S. Army unit, 15 months in Iraq (March 2007 to May 2008) | Multiple-component intervention for deployed unit included 4 phases: predeployment (suicide risk recognition and response training, early identification, and resiliency training for soldiers and families), deployment (education, suicide prevention review board and suicide risk management teams, unit behavioral health needs assessment, unit behavioral health advocates, incident response, and trend monitoring), redeployment (education, postdeployment health assessment, and risk stratification), and reintegration (complete redeployment tasks, prepare for reuniting with families, and address postdeployment health issues). | Suicide rate (per 100,000): 16.0 for intervention unit during the deployment cycle, 24.0 for service members in theater (19.2 across the U.S. Army) | NAb |
| While et al., 2012 (54) | Before-after | 12,881 suicide deaths in mental health services in Wales and England, 1997–2006 | Assessed 9 of the 12 key service recommendations from the English Suicide Prevention Strategy: removing ligature points on inpatient wards, assertive outreach, 24-hour crisis team, follow-up after psychiatric discharge within 7 days, written policy on response to patients noncompliant with treatment, written policy on management of patients with co-occurring mental and substance use disorders, criminal justice sharing, multidisciplinary review and sharing information with families after suicide, and clinical training about suicide risk for staff | Suicide rates that declined pre- versus postintervention (per 10,000 per year): 24-hour crisis care, 11.44 versus 9.32 (p<.001); multidisciplinary review, 11.51 versus 11.39 (p<.001); and co-occurring disorders, 10.51 versus 9.61 (p<.001). | NAb |
a
The outcome evaluated in each study was suicide.
b
Risk-of-bias criteria not applicable (NA) for this study design.
Suicide rates were lower after interventions in six observational studies, including studies of the Air Force Suicide Prevention Program (49,55), a program for an Army Infantry Division deployed to Iraq (50), and studies of police officers (52), university students (51), and health systems (47,54,56). Suicide rates were not lower in two studies of community programs (48,53). However, interventions differed across studies, and all studies were limited by risk of bias inherent to nonrandomized study designs—particularly, that potential confounders were not considered and groups may not have been comparable. Strength-of-evidence grades were low for suicide outcomes (eight observational studies), insufficient for suicide attempt outcomes (no studies), and insufficient for adverse effects (no studies) [see table in online supplement ].
Interventions directed toward individuals.
Ten RCTs of the effectiveness of individual-level health care interventions met inclusion criteria (Table 2) (57–67). Two studies evaluated interventions for secondary prevention (59,64), and the others evaluated interventions for tertiary prevention. Most trials compared usual care with various types of psychotherapy, including cognitive-behavioral therapy, dialectical behavior therapy, personal construct psychotherapy, and problem-solving therapy. One trial examined collaborative care (57,64), and another examined day hospital attendance (59). Participants included outpatient military personnel and nonmilitary psychiatric inpatients or patients at acute risk of suicide. No studies specifically evaluated adverse effects of the interventions, and none specifically considered the effects of pharmacotherapy administered outside the study protocol during the study.
| Study | Population | Intervention and comparison | Outcome | Results | Risk-of-bias rating |
|---|---|---|---|---|---|
| Comtois et al., 2011 (58) | 32 adults ages 19–62, 62% female, with recent suicide attempt or imminent risk; U.S. | CAMSa (patients identify the causes of suicidal ideation and the reduction in suicidal ideation and behavior as a coping strategy; 4–12 sessions lasting 50–60 minutes with CAMS clinicians) versus enhanced usual care (intake with psychiatrist, 1–11 visits with case manager, and medication management as needed) | Suicide attempts and Self-Injury Count score | Suicide attempts and self-inflicted injuries at 12-month follow-up (M±SD): CAMS, 1.2±3.9; enhanced usual care, 3.3±7.6 | High |
| Gallo et al., 2007 (64);Alexopoulos et al., 2009 (57) | 599 adults ages ≥60, 72% female, with score >20 on the Centers for Epidemiologic Studies Depression Scale | Intervention (on-site depression care manager working with primary care physicians to provide algorithm-based care) versus usual care (educational sessions for primary care physicians and notification of patients’ depression but no specific recommendations for individual patients except for psychiatric emergencies) | Suicide | N of suicides at 2-year follow-up and N per 1,000 person-years: intervention, 1 and .7 (95% CI=.0–4.2); usual care, 0 and .0 (95% CI=.0–3.3); N of suicide attempts at 2-year follow-up: intervention, 2; usual care, 3 | Unclear |
| Jones et al., 2008 (59) | 206 patients admitted to an adult psychiatric ward from 1999 to 2002, 52% female; London | Day hospital (attendance expected 9:30 a.m. to 4:30 p.m., with drop-in service on weekends; emphasis on group activities) versus inpatient (conventional psychiatric care and limited program of daily activities) | Suicide | 1 suicide in the day hospital group and 1 in the inpatient group at 12 months postdischarge | High |
| Linehan et al., 2006 (65) | 111 women ages 18–45 with borderline personality disorder and current and past suicidal behaviorsc | DBTb (cognitive-behavioral treatment for suicidal women meeting criteria for borderline personality disorder; treatment targets suicidal behavior, behaviors interfering with treatment delivery, and other severe behaviors for 1 year) versus community treatment by experts (usual care, with treatment uncontrolled by the research team) | Suicide attempts and Suicide Attempt Self-Injury Interview | Suicide attempts: DBT, 23%; community treatment by experts, 46% (p=.01, hazard ratio [HR]=2.66, p=.005) | Unclear |
| Linehan et al., 2015 (60) | 99 adult women with borderline personality disorder with ≥2 suicide attempts or nonsuicidal self-injury acts within 5 years from 2004 to 2010; Seattle | Standard DBT (weekly individual therapy and group skills training, a therapist consultation team, and between-session telephone coaching as needed) versus DBT with skills training (provided group skills training, removed individual component and replaced it with case management) versus DBT with individual therapy (eliminated all skills training and added an activity-based support group) | Suicide and suicide attempts | 1 suicide in standard DBT group; no differences in suicide attempts | Unclear |
| McAuliffe et al., 2014 (61) | 433 psychiatric patients ages 18–64, 65% female, in emergency or inpatient units who reported self-harm within past 3 days; Ireland | Problem-solving skills training (six 2-hour sessions of manualized interpersonal problem-solving skills training) versus usual care (assessment and mental health or crisis services referral) | Suicide | 1 suicide in problem-solving skills training versus 2 in usual care at 12-month follow-up | High |
| McMain et al., 2012 (62) | 180 adults with borderline personality disorder; Toronto | DBT (comprehensive multicomponent intervention for individuals with high suicide risk; contains 4 weekly components, including individual therapy, group skills training, therapist consultation, and as-needed between-session telephone coaches for 1 year) versus general psychiatric management (psychodynamic psychotherapy, case management, and pharmacotherapy for 1 year) | Suicide attempts | At 36-months follow-up, no differences between groups (p=.83). | Unclear |
| Rudd et al., 2015 (63) | 152 active duty Army personnel, 88% male; Fort Carson, Colorado | Brief outpatient cognitive-behavioral therapy (12 sessions, 1–2 weeks apart; first session 90 minutes and following sessions 60 minutes; 3 phases included assessment, cognitive strategies to reduce beliefs and assumptions that serve suicidal thoughts, and relapse prevention) versus usual care (treatment as usual) | Suicide and Suicide Attempt Self-Injury Interview | After 2 years of follow-up, ≥1 suicide attempt by 8 individuals in therapy versus 18 in usual care (14% versus 40%, p=.02); multivariate Cox regression controlled for baseline risk: HR=.31 (95% CI=.13–.75). | Unclear |
| Stewart et al., 2009 (66) | 32 adults ages 20–58, 53% female, receiving inpatient treatment for suicide attempts | Cognitive-behavioral therapy (7 sessions, 1 hour each) versus problem-solving therapy (4 sessions, 1 hour each) versus treatment as usual (usual care provided by local hospital) | Repeated suicide attempts | Average N of suicide attempts (M±SD): cognitive-behavioral therapy, .22±.64, versus usual care, .22±.50 (not significant); problem-solving therapy, .33±.63, versus usual care, .22±.50 (not significant) | High |
| Winter et al., 2007 (67) | 64 adults, 53% female, receiving emergency care following self-harm | Personal construct psychotherapy (2–22 sessions [mean 10.38]), therapeutic techniques appropriate to particular personal construct formulations of the patient’s self-harm as set out in a brief manual) versus usual care (assessment and possible follow-up appointments with a mental health team) | Suicide | 1 suicide for therapy intervention versus 2 for usual care | High |
a
CAMS, Collaborative Assessment and Management of Suicidality
b
DBT, dialectical behavior therapy
c
Two suicide attempts or self-injuries within the past five years, with one in past eight weeks
Risk of bias was rated high in five trials (58,59,61,66,67) and unclear in five (57,60,62–65). Most studies were underpowered to detect differences between treatment and usual care. Additional limitations included lack of information on randomization (59,66,67), allocation concealment (58–60,63,65–67), and blinding (58,59,64,66,67); lack of outcome reporting (59,61,67); and unclear or lack of specified outcome measures (60,63,66).
Only two of ten trials reported statistically significant differences between treatment and usual care (63,65), although risk of bias was unclear for both trials. In a trial of outpatient active duty soldiers with recent suicide attempts or ideation, those in a brief cognitive-behavioral therapy program were less likely to have made suicide attempts at two-year follow-up than those in usual care (14% versus 40%, p=.02; hazard ratio=.38, 95% CI=.16–.87) (63). In a trial involving women with borderline personality disorder, those receiving dialectical behavior therapy had fewer suicide attempts than those receiving usual care at one-year follow-up (23% versus 46%, p=.01) (65). Strength-of-evidence grades were low for suicide attempt outcomes (seven trials), insufficient for suicide outcomes (four trials), and insufficient for adverse effects (no studies).
Discussion
Nineteen studies of risk assessment methods to identify individuals at increased risk of suicide or suicide attempts evaluated 19 different approaches. Results indicated fair or better diagnostic accuracy for most methods. However, methodological limitations in many of the studies threatened the validity and applicability of the findings.
Prediction models derived from large patient databases may provide a more rigorous approach to risk assessment than other methods. Although some models demonstrated fair to good diagnostic accuracy in single studies with low risk of bias, they have not yet been implemented and further tested in clinical settings. Several remaining concerns include their feasibility in clinical practice, impact on workflow, and incremental value in identifying patients for further intervention. For example, although results indicated high predictive accuracy in a study of the Army STARRS risk algorithm, this method incorporated information from 38 administrative databases, including data that may not or cannot be obtained prospectively. Whether key information in the model would be routinely kept up to date and available to inform clinical practice in real time is unclear. A new model based on Army STARRS administrative data from outpatient mental health visits, published after the systematic review searches, may provide a more feasible approach (68). Another approach, using responses to the suicidal ideation question from the nine-item Patient Health Questionnaire (PHQ-9), a measure frequently used in clinical practice, was recently found to be associated with up to a 75% increased risk of suicide in a sample of nearly 400,000 VA patients (69).
Studies of various clinician-rated or patient–self-reported risk assessment instruments and scales indicated accuracy that varied across methods and cut-points. Some instruments may provide diagnostic value to specific patient subgroups, such as those with previous suicide attempts or co-occurring conditions. Although these methods may be useful, studies evaluating them are currently inconclusive and limited by small sample sizes, methodological limitations, and unclear applicability to clinical practice. These deficiencies also limit comparisons between types of risk assessments, patient subgroups, settings, or other factors. Although these methods appear in general to have lower accuracy than more complex models based on electronic records or administrative data, the accuracy of the two approaches was compared in only one study (41).
One of the biggest challenges in using methods to identify suicide risk in clinical settings is the low incidence rate of suicide, especially in primary care or ambulatory mental health care populations. Our results revealed several methods with reasonably high sensitivity. However, positive predictive values were much lower because, unlike sensitivity, predictive values are affected by the low incidence of suicide. It is unclear at this point what level of risk would be needed for a specific method to be clinically useful. For example, even with a positive predictive value of 30% for suicidal behavior, as observed in included studies of high-risk veteran populations (30,31), 70 of every 100 cases would be false positives. Moreover, the false-positive rate becomes exponentially higher when the outcome is suicide itself, even in high-risk populations. The false-positive rate is even higher when suicide risk assessment is applied broadly across unselected populations. For example, if using the ReACT Self-Harm Rule with a sensitivity of 88% and specificity of 24% for assessing risk among all veterans receiving care in VA facilities (suicide rate of 27.6 per 100,000) (11), the positive predictive value would be only 3%. Given that this review found no studies of adverse effects, such as involuntary hospitalization or other actions that might be triggered by a positive risk assessment, the inadequacy of the current evidence base to offer practical guidance to clinicians becomes even more apparent.
Future research should build on current work. Risk models should be further refined, and those demonstrating fair to good diagnostic accuracy, such as the ReACT Self-Harm Rule, should be prospectively evaluated in large clinical populations, preferably through cluster-randomized trials. Studies should also assess the effectiveness and potential adverse effects of risk prediction tools in clinical care pathways designed to prevent suicide or suicide attempts. To ensure their clinical applicability to VA and military populations, methods should be specifically evaluated in these groups.
Future research should also test novel approaches to suicide risk assessment that build on existing developmental work. Although many studies of risk assessment methods have relied on patient self-reported data, a study of the computer-administered Implicit Association Test is an important exception (38). This test uses individuals’ reaction times when classifying semantic stimuli to predict future suicide attempts. In addition, individuals with histories of suicide attempts exhibit certain patterns of cognitive deficits that can be detected by tasks commonly included in neuropsychological testing batteries (70,71). Exploratory studies examining biological markers for suicide or suicidal behaviors (72–74) or neuroimaging to identify associations with suicide-related outcomes (75,76) could be further expanded to determine their roles in clinical care and among specific patient subgroups. Ultimately, risk assessment approaches should be brief and easily administered for clinical applications across multiple settings.
Eight studies of the effectiveness of population-level interventions and ten RCTs of individual-level health care interventions met inclusion criteria for this review. However, results were inconclusive because interventions varied across studies, risk of bias could not be assessed for many studies because of study design, comparability of groups was not established, and most studies were underpowered to detect differences between intervention and comparison groups. In addition, high-risk patients may have been excluded from studies because of their need for acute care, and adverse effects were not reported.
Studies of population-level health care service interventions for suicide prevention should be conducted in additional populations to further validate results of the initial studies and to demonstrate the programs’ applicability to general clinical practice. Many of the intervention studies compared outcomes before and after the interventions were implemented, raising concerns regarding regression to the mean and cohort effects. Although RCTs may be impractical on a population level, more robust observational study designs, such as interrupted time-series analysis, would strengthen the evidence base. Additional details about how the program components were implemented and maintained in practice are also necessary to establish portable service packages and translate this work to other settings. Restricting access to lethal means is an effective method of suicide prevention (7,77), although no recently published studies of health care services using this approach met inclusion criteria for this review. Examples of current efforts in the VA to reduce access to lethal means include an ongoing study of blister packaging for medications (78) and distribution of gun locks. Studies examining efforts to counsel veterans on firearms safety and delaying or restricting access to firearms have been published (79,80), but more work is needed to help guide clinician discussions with veterans and to establish the effectiveness of such strategies for VA health care settings.
Studies of individual-level interventions generally targeted individuals identified as at high risk of suicide because of recent suicide attempts or self-harm or existence of psychiatric conditions. Although these approaches are important, no studies tested interventions to increase protective factors, such as social integration (81,82). Improvement of this evidence base will require larger, more rigorous RCTs of existing interventions as well as innovative approaches, such as novel uses of technology to support or enhance care for individuals at risk of suicide (83,84), safety planning in health settings (85–87), and peer support.
This review was limited by its focus on veteran and military populations in the United States, which may have excluded important studies. However, this focus improved its clinical relevance to this population, as well as other adults with similar demographic characteristics. Of the 37 studies included in this review, six studies of risk assessment and three studies of interventions specifically included veterans or military personnel. Also, this review included only English-language articles, which could result in language bias, although studies not published in English may be less relevant to the target population. Studies were not available for some key questions, including comparisons of the accuracy of risk methods by settings, delivery modes, and targeted populations. One of the useful measures of test performance—positive predictive value—could not be summarized and compared because it was often not included in studies. Data on adverse effects of risk assessment and prevention interventions were also absent across studies.
Conclusions
Risk assessment methods have been shown to be sensitive predictors of subsequent suicide and suicide attempts, but the frequency of false positives limits their clinical utility. Future research should continue to refine these methods and examine clinical applications. Studies of suicide prevention interventions provide inconclusive evidence to support their use, and additional RCTs of promising individual therapies and site-randomized population-level interventions are needed.
Footnote
The U.S. Department of Veterans Affairs had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication. The findings and conclusions reported here are those of the authors, who are responsible for its contents, and do not necessarily represent the views of the U.S. Department of Veterans Affairs or the U.S. government. No statement in this article should be construed as an official position of the U.S. Department of Veterans Affairs.
Supplementary Material
File (appi.ps.201600384.ds001.pdf)
- View/Download
- 140.41 KB
References
1.
Suicide: Facts at a Glance. Atlanta, Centers for Disease Control and Prevention, Injury Prevention and Control, Division of Violence Prevention, 2015
2.
LeFevre ML, US Preventive Services Task Force: Screening for suicide risk in adolescents, adults, and older adults in primary care: US Preventive Services Task Force recommendation statement. Annals of Internal Medicine 160:719–726, 2014
3.
Gaynes BN, West SL, Ford CA, et al: Screening for suicide risk in adults: a summary of the evidence for the US Preventive Services Task Force. Annals of Internal Medicine 140:822–835, 2004
4.
Detecting and Treating Suicide Ideation in All Settings. Sentinel Event Alert 56. Chicago, Joint Commission, 2016
5.
Ahmedani BK, Simon GE, Stewart C, et al: Health care contacts in the year before suicide death. Journal of General Internal Medicine 29:870–877, 2014
6.
O’Neil M, Peterson K, Low A, et al: Suicide Prevention Interventions and Referral/Follow-Up Services: A Systematic Review. Washington, DC, US Department of Veterans Affairs, 2012. http://www.hsrd.research.va.gov/publications/esp/suicide-interventions.cfm
7.
Shekelle P, Bagley S, Munjas B: Strategies for Suicide Prevention in Veterans. Washington, DC, US Department of Veterans Affairs, 2009. http://www.hsrd.research.va.gov/publications/esp/suicide.cfm
8.
Kang HK, Bullman TA, Smolenski DJ, et al: Suicide risk among 1.3 million veterans who were on active duty during the Iraq and Afghanistan wars. Annals of Epidemiology 25:96–100, 2015
9.
Swanson JW, Bonnie RJ, Appelbaum PS: Getting serious about reducing suicide: more “how” and less “why.” JAMA 314:2229–2230, 2015
10.
Schoenbaum M, Kessler RC, Gilman SE, et al: Predictors of suicide and accident death in the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS): results from the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). JAMA Psychiatry 71:493–503, 2014
11.
Hoffmire CA, Kemp JE, Bossarte RM: Changes in suicide mortality for veterans and nonveterans by gender and history of VHA service use, 2000–2010. Psychiatric Services 66:959–965, 2015
12.
Hawton K, Witt KG, Salisbury TL, et al: Psychosocial interventions following self-harm in adults: a systematic review and meta-analysis. Lancet Psychiatry 3:740–750, 2016
13.
Meerwijk EL, Parekh A, Oquendo MA, et al: Direct versus indirect psychosocial and behavioural interventions to prevent suicide and suicide attempts: a systematic review and meta-analysis. Lancet Psychiatry 3:544–554, 2016
14.
Pirkis J, Too LS, Spittal MJ, et al: Interventions to reduce suicides at suicide hotspots: a systematic review and meta-analysis. Lancet Psychiatry 2:994–1001, 2015
15.
Zalsman G, Hawton K, Wasserman D, et al: Suicide prevention strategies revisited: 10-year systematic review. Lancet Psychiatry 3:646–659, 2016
16.
Turecki G, Brent DA: Suicide and suicidal behaviour. Lancet 387:1227–1239, 2016
17.
Nelson H: Systematic Reviews to Answer Health Care Questions. Philadelphia, Wolters Kluwer Health, 2014
18.
Nelson H, Low A, Denneson L, et al: Systematic Review of Suicide Prevention in US Military Veterans. PROSPERO: CRD42015019089. York, United Kingdom, PROSPERO International Prospective Register of Systematic Reviews, 2015. http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42015019089
19.
Nelson H, Denneson L, Low A, et al: Systematic Review of Suicide Prevention in Veterans. VA ESP Project 05-225. Washington, DC, US Department of Veterans Affairs, 2015. http://www.hsrd.research.va.gov/publications/esp/suicideprevention.cfm
20.
Crosby A, Ortega L, Melanson C: Self-Directed Violence Surveillance: Uniform Definitions and Recommended Data Elements, Version 1.0. Atlanta, Centers for Disease Control and Prevention, National Center for Injury Prevention and Control, 2011
21.
Memorandum re: Standardized Suicide Nomenclature (Self-Directed Violence Classification System). Washington, DC, US Department of Veterans Affairs, 2010
22.
Haney E, O’Neil M, Carson S, et al: Suicide Risk Factors and Risk Assessment Tools: A Systematic Review. Washington, DC, US Department of Veterans Affairs, 2012. http://www.hsrd.research.va.gov/publications/esp/suicide-risk.cfm
23.
Higgins J, Green S: Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. London, Cochrane Collaboration, 2011. http://handbook.cochrane.org
24.
Harris RP, Helfand M, Woolf SH, et al: Current methods of the US Preventive Services Task Force: a review of the process. American Journal of Preventive Medicine 20(suppl):21–35, 2001
25.
US Preventive Services Task Force Procedure Manual. AHRQ pub no 08-05118-EF. Rockville, Md, Agency for Healthcare Research and Quality, 2008. http://www.uspreventiveservicestaskforce.org/uspstf08/methods/procmanual.htm
26.
Hayden JA, Côté P, Bombardier C: Evaluation of the quality of prognosis studies in systematic reviews. Annals of Internal Medicine 144:427–437, 2006
27.
Berkman N, Lohr K, Ansari M, et al: Grading the Strength of a Body of Evidence When Assessing Health Care Interventions for the Effective Health Care Program of the Agency for Healthcare Research and Quality: An Update. AHRQ pub no 13(14)-EHC130-EF. Rockville, Md, Agency for Healthcare Research and Quality, 2013. http://www.effectivehealthcare.ahrq.gov/ehc/products/457/1752/methods-guidance-grading-evidence-131118.pdf
28.
Breshears RE, Brenner LA, Harwood JE, et al: Predicting suicidal behavior in veterans with traumatic brain injury: the utility of the personality assessment inventory. Journal of Personality Assessment 92:349–355, 2010
29.
Hartl TL, Rosen C, Drescher K, et al: Predicting high-risk behaviors in veterans with posttraumatic stress disorder. Journal of Nervous and Mental Disease 193:464–472, 2005
30.
Hendin H, Al Jurdi RK, Houck PR, et al: Role of intense affects in predicting short-term risk for suicidal behavior: a prospective study. Journal of Nervous and Mental Disease 198:220–225, 2010
31.
Tiet QQ, Ilgen MA, Byrnes HF, et al: Suicide attempts among substance use disorder patients: an initial step toward a decision tree for suicide management. Alcoholism, Clinical and Experimental Research 30:998–1005, 2006
32.
Bernert RA, Turvey CL, Conwell Y, et al: Association of poor subjective sleep quality with risk for death by suicide during a 10-year period: a longitudinal, population-based study of late life. JAMA Psychiatry 71:1129–1137, 2014
33.
Bolton JM, Spiwak R, Sareen J: Predicting suicide attempts with the SAD PERSONS scale: a longitudinal analysis. Journal of Clinical Psychiatry 73:e735–e741, 2012
34.
Galfalvy HC, Oquendo MA, Mann JJ: Evaluation of clinical prognostic models for suicide attempts after a major depressive episode. Acta Psychiatrica Scandinavica 117:244–252, 2008
35.
Galynker I, Yaseen ZS, Briggs J, et al: Attitudes of acceptability and lack of condemnation toward suicide may be predictive of post-discharge suicide attempts. BMC Psychiatry 15:87, 2015
36.
Kessler RC, Warner CH, Ivany C, et al: Predicting suicides after psychiatric hospitalization in US Army soldiers: the Army Study To Assess Risk and rEsilience in Servicemembers (Army STARRS). JAMA Psychiatry 72:49–57, 2015
37.
McCarthy JF, Bossarte RM, Katz IR, et al: Predictive modeling and concentration of the risk of suicide: implications for preventive interventions in the US Department of Veterans Affairs. American Journal of Public Health 105:1935–1942, 2015
38.
Nock MK, Park JM, Finn CT, et al: Measuring the suicidal mind: implicit cognition predicts suicidal behavior. Psychological Science 21:511–517, 2010
39.
Rana S, Gupta S, Venkatesh S, et al: An Analysis of Suicide Risk Assessment. Report no TR-PRaDA-03/12. Geelong, Victoria, Australia, Pattern Recognition and Data Analytics, 2012. https://www.academia.edu/13879097/An_Analysis_of_Suicide_Risk_Assessment?auto=download
40.
Steeg S, Kapur N, Webb R, et al: The development of a population-level clinical screening tool for self-harm repetition and suicide: the ReACT Self-Harm Rule. Psychological Medicine 42:2383–2394, 2012
41.
Tran T, Luo W, Phung D, et al: Risk stratification using data from electronic medical records better predicts suicide risks than clinician assessments. BMC Psychiatry 14:76, 2014
42.
van Spijker BA, Batterham PJ, Calear AL, et al: The Suicidal Ideation Attributes Scale (SIDAS): community-based validation study of a new scale for the measurement of suicidal ideation. Suicide and Life-Threatening Behavior 44:408–419, 2014
43.
Yaseen ZS, Fisher K, Morales E, et al: Love and suicide: the structure of the Affective Intensity Rating Scale (AIRS) and its relation to suicidal behavior. PLoS One 7:e44069, 2012
44.
Yaseen ZS, Gilmer E, Modi J, et al: Emergency room validation of the revised Suicide Trigger Scale (STS-3): a measure of a hypothesized suicide trigger state. PLoS One 7:e45157, 2012
45.
Yaseen ZS, Kopeykina I, Gutkovich Z, et al: Predictive validity of the Suicide Trigger Scale (STS-3) for post-discharge suicide attempt in high-risk psychiatric inpatients. PLoS One 9:e86768, 2014
46.
Yen S, Shea MT, Walsh Z, et al: Self-harm subscale of the Schedule for Nonadaptive and Adaptive Personality (SNAP): predicting suicide attempts over 8 years of follow-up. Journal of Clinical Psychiatry 72:1522–1528, 2011
47.
Coffey CE: Building a system of perfect depression care in behavioral health. Joint Commission Journal on Quality and Patient Safety 33:193–199, 2007
48.
Harris MG, Burgess PM, Chant DC, et al: Impact of a specialized early psychosis treatment programme on suicide: retrospective cohort study. Early Intervention in Psychiatry 2:11–21, 2008
49.
Knox KL, Pflanz S, Talcott GW, et al: The US Air Force suicide prevention program: implications for public health policy. American Journal of Public Health 100:2457–2463, 2010
50.
Warner CH, Appenzeller GN, Parker JR, et al: Suicide prevention in a deployed military unit. Psychiatry 74:127–141, 2011
51.
Joffe P: An empirically supported program to prevent suicide in a college student population. Suicide and Life-Threatening Behavior 38:87–103, 2008
52.
Mishara BL, Martin N: Effects of a comprehensive police suicide prevention program. Crisis 33:162–168, 2012
53.
Walrath C, Garraza LG, Reid H, et al: Impact of the Garrett Lee Smith youth suicide prevention program on suicide mortality. American Journal of Public Health 105:986–993, 2015
54.
While D, Bickley H, Roscoe A, et al: Implementation of mental health service recommendations in England and Wales and suicide rates, 1997–2006: a cross-sectional and before-and-after observational study. Lancet 379:1005–1012, 2012
55.
Knox KL, Litts DA, Talcott GW, et al: Risk of suicide and related adverse outcomes after exposure to a suicide prevention programme in the US Air Force: cohort study. BMJ 327:1376, 2003
56.
Appleby L, Shaw J, Sherratt J, et al: Safety First: Five-Year Report of the National Confidential Inquiry Into Suicide and Homicide by People With Mental Illness. London, Stationery Office, 2001
57.
Alexopoulos GS, Reynolds CF III, Bruce ML, et al: Reducing suicidal ideation and depression in older primary care patients: 24-month outcomes of the PROSPECT study. American Journal of Psychiatry 166:882–890, 2009
58.
Comtois KA, Jobes DA, O’Connor S, et al: Collaborative Assessment and Management of Suicidality (CAMS): feasibility trial for next-day appointment services. Depression and Anxiety 28:963–972, 2011
59.
Jones G, Gavrilovic JJ, McCabe R, et al: Treating suicidal patients in an acute psychiatric day hospital: a challenge to assumptions about risk and overnight care. Journal of Mental Health 17:375–387, 2008
60.
Linehan MM, Korslund KE, Harned MS, et al: Dialectical behavior therapy for high suicide risk in individuals with borderline personality disorder: a randomized clinical trial and component analysis. JAMA Psychiatry 72:475–482, 2015
61.
McAuliffe C, McLeavey BC, Fitzgerald T, et al: Group problem-solving skills training for self-harm: randomised controlled trial. British Journal of Psychiatry 204:383–390, 2014
62.
McMain SF, Guimond T, Streiner DL, et al: Dialectical behavior therapy compared with general psychiatric management for borderline personality disorder: clinical outcomes and functioning over a 2-year follow-up. American Journal of Psychiatry 169:650–661, 2012
63.
Rudd MD, Bryan CJ, Wertenberger EG, et al: Brief cognitive-behavioral therapy effects on post-treatment suicide attempts in a military sample: results of a randomized clinical trial with 2-year follow-up. American Journal of Psychiatry 172:441–449, 2015
64.
Gallo JJ, Bogner HR, Morales KH, et al: The effect of a primary care practice-based depression intervention on mortality in older adults: a randomized trial. Annals of Internal Medicine 146:689–698, 2007
65.
Linehan MM, Comtois KA, Murray AM, et al: Two-year randomized controlled trial and follow-up of dialectical behavior therapy vs therapy by experts for suicidal behaviors and borderline personality disorder. Archives of General Psychiatry 63:757–766, 2006
66.
Stewart CD, Quinn A, Plever S, et al: Comparing cognitive behavior therapy, problem solving therapy, and treatment as usual in a high risk population. Suicide and Life-Threatening Behavior 39:538–547, 2009
67.
Winter D, Sireling L, Riley T, et al: A controlled trial of personal construct psychotherapy for deliberate self-harm. Psychology and Psychotherapy 80:23–37, 2007
68.
Kessler RC, Stein MB, Petukhova MV, et al: Predicting suicides after outpatient mental health visits in the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). Molecular Psychiatry 22:544–551, 2017
69.
Louzon SA, Bossarte R, McCarthy JF, et al: The clinical utility of PHQ-9 item 9 for suicide prediction: in reply. Psychiatric Services 67:1042–1043, 2016
70.
McGirr A, Dombrovski AY, Butters MA, et al: Deterministic learning and attempted suicide among older depressed individuals: cognitive assessment using the Wisconsin Card Sorting Task. Journal of Psychiatric Research 46:226–232, 2012
71.
Richard-Devantoy S, Jollant F, Kefi Z, et al: Deficit of cognitive inhibition in depressed elderly: a neurocognitive marker of suicidal risk. Journal of Affective Disorders 140:193–199, 2012
72.
Galfalvy H, Huang YY, Oquendo MA, et al: Increased risk of suicide attempt in mood disorders and TPH1 genotype. Journal of Affective Disorders 115:331–338, 2009
73.
Kim YK, Hwang JA, Lee HJ, et al: Association between norepinephrine transporter gene (SLC6A2) polymorphisms and suicide in patients with major depressive disorder. Journal of Affective Disorders 158:127–132, 2014
74.
Shinozaki G, Romanowicz M, Passov V, et al: State dependent gene-environment interaction: serotonin transporter gene–child abuse interaction associated with suicide attempt history among depressed psychiatric inpatients. Journal of Affective Disorders 147:373–378, 2013
75.
Matthews S, Spadoni A, Knox K, et al: Combat-exposed war veterans at risk for suicide show hyperactivation of prefrontal cortex and anterior cingulate during error processing. Psychosomatic Medicine 74:471–475, 2012
76.
Serafini G, Pompili M, Innamorati M, et al: Affective temperamental profiles are associated with white matter hyperintensity and suicidal risk in patients with mood disorders. Journal of Affective Disorders 129:47–55, 2011
77.
Mann JJ, Apter A, Bertolote J, et al: Suicide prevention strategies: a systematic review. JAMA 294:2064–2074, 2005
78.
Blister Packaging Medication to Increase Treatment Adherence and Clinical Response. NCT01118208. Washington, DC, ClinicalTrials.gov, 2015. http://ClinicalTrials.gov/show/NCT01118208
79.
Walters H, Kulkarni M, Forman J, et al: Feasibility and acceptability of interventions to delay gun access in VA mental health settings. General Hospital Psychiatry 34:692–698, 2012
80.
Britton PC, Bryan CJ, Valenstein M: Motivational interviewing for means restriction counseling with patients at risk for suicide. Cognitive and Behavioral Practice 23:51–61, 2016
81.
Tsai AC, Lucas M, Sania A, et al: Social integration and suicide mortality among men: 24-year cohort study of US health professionals. Annals of Internal Medicine 161:85–95, 2014
82.
Tsai AC, Lucas M, Kawachi I: Association between social integration and suicide among women in the United States. JAMA Psychiatry 72:987–993, 2015
83.
Berrouiguet S, Gravey M, Le Galudec M, et al: Post-acute crisis text messaging outreach for suicide prevention: a pilot study. Psychiatry Research 217:154–157, 2014
84.
Berrouiguet S, Alavi Z, Vaiva G, et al: SIAM (Suicide Intervention Assisted by Messages): the development of a post-acute crisis text messaging outreach for suicide prevention. BMC Psychiatry 14:294, 2014
85.
Ghahramanlou-Holloway M, Brown GK, Currier GW, et al: Safety Planning for Military (SAFE MIL): rationale, design, and safety considerations of a randomized controlled trial to reduce suicide risk among psychiatric inpatients. Contemporary Clinical Trials 39:113–123, 2014
86.
Currier GW, Brown GK, Brenner LA, et al: Rationale and study protocol for a two-part intervention: safety planning and structured follow-up among veterans at risk for suicide and discharged from the emergency department. Contemporary Clinical Trials 43:179–184, 2015
87.
Knox KL, Stanley B, Currier GW, et al: An emergency department-based brief intervention for veterans at risk for suicide (SAFE VET). American Journal of Public Health 102(suppl 1):S33–S37, 2012
Information & Authors
Information
Published In
Cover: Birmingham Breakdown #2, by Craig Moran, 2010. Oil on canvas. Collection of the artist, Washington, D.C.
History
Received: 19 August 2016
Revision received: 14 December 2016
Revision received: 31 January 2017
Accepted: 10 March 2017
Published online: 15 June 2017
Published in print: October 01, 2017
Keywords
Authors
Competing Interests
The authors report no financial relationships with commercial interests.
Funding Information
Quality Enhancement Research Initiative10.13039/100007181
U.S. Department of Veterans Affairs10.13039/100000738
Office of Research and Development10.13039/100006379
Veterans Health Administration
This work was funded by the Quality Enhancement Research Initiative, Office of Research and Development, Veterans Health Administration (VHA), U.S. Department of Veterans Affairs. Dr. O’Neil’s work was supported in part by PCOR K12 award 1 K12 HS019456 01 from the Agency for Healthcare Research and Quality. Dr. Teo’s work was supported in part by career development award CDA 14-428 from the VHA’s Health Service Research and Development.Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).