A Risk Calculator to Predict Suicide Attempts Among Individuals With Early-Onset Bipolar Disorder
Abstract
Objectives:
To build a one-year risk calculator (RC) to predict individualized risk for suicide attempt in early-onset bipolar disorder.
Methods:
Youth numbering 394 with bipolar disorder who completed ≥2 follow-up assessments (median follow-up length = 13.1 years) in the longitudinal Course and Outcome of Bipolar Youth (COBY) study were included. Suicide attempt over follow-up was assessed via the A-LIFE Self-Injurious/Suicidal Behavior scale. Predictors from the literature on suicidal behavior in bipolar disorder that are readily assessed in clinical practice were selected and trichotomized as appropriate (presence past 6 months/lifetime history only/no lifetime history). The RC was trained via boosted multinomial classification trees; predictions were calibrated via Platt scaling. Half of the sample was used to train, and the other half to independently test the RC.
Results:
There were 249 suicide attempts among 106 individuals. Ten predictors accounted for >90% of the cross-validated relative influence in the model (AUC = 0.82; in order of relative influence): (1) age of mood disorder onset; (2) non-suicidal self-injurious behavior (trichotomized); (3) current age; (4) psychosis (trichotomized); (5) socioeconomic status; (6) most severe depressive symptoms in past 6 months (trichotomized none/subthreshold/threshold); (7) history of suicide attempt (trichotomized); (8) family history of suicidal behavior; (9) substance use disorder (trichotomized); (10) lifetime history of physical/sexual abuse. For all trichotomized variables, presence in the past 6 months reliably predicted higher risk than lifetime history.
Conclusions:
This RC holds promise as a clinical and research tool for prospective identification of individualized high-risk periods for suicide attempt in early-onset bipolar disorder.
Reprinted from Bipolar Disord 2022; 24:749–757, with permission from John Wiley and Sons. Copyright © 2022
INTRODUCTION
Suicide is currently the second leading cause of death worldwide among youth aged 12–24 years.1 The majority of youth who die by suicide meet criteria for at least one psychiatric disorder; of these, bipolar disorder confers greatest risk. Nearly one-third of youth with bipolar disorder attempt suicide at least once by mid-adolescence,2 60% attempt at least once in their lifetimes, and up to 20% will die by suicide, rendering the risk of death by suicide 60 times higher among individuals with bipolar disorder than that of the general population.3
Data from cross-sectional, prospective, and meta-analytic studies identify specific factors associated with elevated risk for suicidal behavior among individuals with bipolar disorder, one of the most potent of which is early age of bipolar illness onset.4 Others include mixed episodes, psychosis, non-suicidal self-injury, family history of suicide, and history of abuse.5–7 Identification of these group-level risk factors is critical, but yield information about risk for the group as a whole. As such, they may not reliably predict risk for any specific individual. Moreover, since these risk factors are both numerous and heterogenous, it is not surprising that a recent large-scale meta-analysis shows that these factors predict risk for suicide attempt only slightly better than chance,8 as do clinical judgment,9 and clinical rating scales.10
Given that suicidal behavior results from a complex interaction of factors, suicide prevention efforts among high-risk groups like individuals with early-onset bipolar disorder could be enhanced by the capacity to predict an individual's risk for SA over a specified time period. A risk calculator (RC) is a clinical tool that uses an optimally identified set of risk factors to compute the probability of a specified event in an individual patient.11 RCs are widely used by physicians to enhance clinical decision making across several health conditions including stroke and cancer.12,13 Recently, RCs have been developed to predict course and treatment response in psychiatric disorders including psychosis11 and depression.14 Our group developed RCs to predict 5-year bipolar disorder onset among offspring of parents with bipolar disorder,15 risk for progression from bipolar, not otherwise specified (NOS), to bipolar-I/-II among youth,16 and risk for mood recurrence among young adults with bipolar disorder.17
Efforts to create RCs to predict individual-level risk for suicide attempt have been limited to date. Spittal et al.18 developed the 4-item Repeated Episodes of Self-Harm (RESH) scale to predict 6-month risk of self-harm repetition (with or without suicidal intent) among hospitalized individuals, with AUC = 0.75. Fazel et al.19 describe the 17-item OxMIS (Oxford Mental Illness and Suicide Tool) to predict suicide death over 1 year among individuals with severe mental illness (AUC = 0.71). These RCs demonstrate moderate predictive power (62%–100% at varying thresholds and 75%, respectively), but low sensitivity (6%–74% and 55%) among varied samples. Prediction may be improved by constructing tailored models within particular groups at higher risk, like individuals with early-onset bipolar disorder. Furthermore, ultimate clinical benefit of RCs is contingent upon inclusion of predictors readily assessed across settings. To highlight, Melhem et al.20 computed a risk score for suicide attempt over 12 years among offspring of parents with mood disorders using six readily assessed clinical factors. The model yielded high sensitivity (87%) and moderate improvement in specificity over the aforementioned tools (63%), but lacks their temporal specificity.
We describe development of a RC to predict an individual's one-year risk for SA for youth with bipolar disorder followed for a median of 13.1 years in the multi-site longitudinal Course and Outcome of Bipolar Youth (COBY) study. To enhance clinical utility, predictors of suicidal behavior in bipolar disorder readily assessed in clinical practice were identified from systematic reviews.
METHODS
We present a brief summary of COBY study methods below. More details are in prior COBY publications.21
Participants
From October 2000 through July 2006, COBY recruited 446 youth aged 7–17 years (mean intake age = 12.6, SD = 3.3, 53% male, 82% White) with a primary diagnosis of DSM-IV bipolar-I, -II, or COBY operationalized Not Otherwise Specified (-NOS; see Psychiatric Diagnosis). Youth with schizophrenia, developmental delay, and mood disorders secondary to substances, medications, or medical illness were excluded. To model the RC, we examined data from 394 participants who completed at least two follow-up assessments (median follow-up = 13.1 years; range = 1–18 years; Table 1); we assessed longitudinal data retrospectively at average follow-up periods of 36 weeks (SD = 18 weeks).
| Follow-up statistics | Median, Mean (SD) |
|---|---|
| Follow-up length in years | 13.13, 12.43 (3.62) |
| Duration between assessments in weeks | 36, 40 (17.73) |
| Intake variables | Mean (SD) or N (%) |
| Age | 12.70 (3.28) |
| Socioeconomic status (SES)a | 3.41 (1.18) |
| Sex at birth (% female) | 186 (47.21%) |
| Race (% White) | 324 (82.23%) |
| Living with both biological parents (%) | 167 (42.39%) |
| Age of mood disorder onset | 9.32 (3.93) |
| BP subtype | |
| I | 236 (59.90%) |
| II | 28 (7.11%) |
| NOSb | 130 (32.99%) |
| Lifetime/Follow-up clinical variables | N (%) |
| Suicide attempt | 184 (46.70%) |
| Suicidal ideation | 312 (79.19%) |
| Non-suicidal self-injurious behavior | 208 (52.79%) |
| Psychiatric hospitalization | 274 (69.54%) |
| Mixed episode | 157 (39.85%) |
| Psychotic symptoms | 146 (37.06%) |
| Sexual/physical abuse | 186 (47.21%) |
| Comorbid disorders | |
| Anxiety disorder | 293 (74.37%) |
| Substance use disorder | 167 (42.39%) |
| Family history | |
| BP | 230 (58.38%) |
| Depression | 347 (88.07%) |
| Anxiety disorder | 290 (73.60%) |
| Substance use disorder | 277 (70.30%) |
| Suicide attempt | 171 (43.40%) |
Abbreviations: BP, bipolar disorder; COBY, course and outcome of bipolar youth.
a
Hollingshead-Redlich Criteria.
b
COBY operationalized criteria.
Procedures
The Institutional Review Boards at the three participating sites approved all study procedures. Prior to study procedure administration, parents and participants provided informed consent. Assessments were conducted by trained master's level staff who maintained reliability on all study assessments.21
Psychiatric diagnosis.
Study assessors interviewed parents and children at study intake to determine current and past psychiatric diagnoses. We used the Schedule for Affective Disorders and Schizophrenia for School-Aged Children--Present and Lifetime Version (K-SADS-PL)22 to assess for non-mood psychiatric disorders, and the K-SADS-P mood sections,23 plus additional items from the K-SADS Mania Rating Scale (MRS),24 to assess mood symptoms. Consensus scores reflect data from youth and parents, and child psychiatrists/psychologists confirmed all diagnoses. KSADS-PL kappas ≥ 0.8 for psychiatric disorders, and K-MRS and KSADS-P depression section intraclass correlation coefficients (ICCs) ≥ 0.95. Eligible participants met criteria for DSM-IV bipolar-I (n = 292), -II (n = 56), or -NOS (n = 65), using research criteria operationalized for COBY.21 Age at mood disorder onset was when the participant first met DSM-IV criteria for any mood episode (i.e., hypo/manic, mixed, major depressive, or bipolar-NOS).
Suicide attempt.
History of suicide attempt was defined as any self-injurious act that reached or exceeded an operationalized threshold (see below) of lethal intent and/or medical lethality, assessed via the K-SADS-P depression section suicidal acts items23 (assessed during the current and most severe past episodes), and/or the K-SADS Summary Lifetime Diagnostic Checklist SA item. Past non-suicidal self-injury was assessed via the K-SADS-P non-suicidal self-damaging acts item (positive if ≥3, infrequent). We utilized the A-LIFE Self-Injurious/Suicidal Behavior Scale25 to document all self-injurious behaviors, regardless of intent, during follow-up. For each self-injurious behavior over the follow-up period, date, method, intent [using K-SADS Depression Scale item ratings23: 1 (none) to 6 (extreme, careful planning, and every expectation of death)] and medical threat [1 (no danger) to 7 (death)] are coded. In keeping with our prior work,25 suicide attempt was defined as any self-injurious behavior that included definite intent (≥3, definite but ambivalent) and/or mild lethality (≥3). All self-injurious behaviors for which intent and lethality were rated minimal or less (≤2) were considered non-suicidal self-injurious behavior.
Demographic variables.
We obtained demographic information via a parent-report form, and history of treatment and abuse via medical history questionnaire. Socioeconomic status (SES) was ascertained using the four-factor Hollingshead scale.26 A demographic update form documented changes over follow-up.
Family psychiatric history.
Study assessors administered the Structured Clinical Interview for DSM-IV27 with the participating parent at intake to gather parental psychiatric history; familial psychiatric history was collected via an enhanced version of the Family History Screen28 and updated at each follow-up assessment. Family history for each condition (including lifetime suicide attempt) was considered positive if any first- or second-degree relative met criteria for ‘probable’ or ‘definite’ disorder.
Psychiatric symptoms.
We documented changes in psychiatric symptoms over follow-up using the Adolescent Longitudinal Interview Follow-Up Evaluation (A-LIFE).19 The A-LIFE Psychiatric Status Rating Scales (PSR) yield week-by-week symptom ratings using numeric values operationally linked to DSM-IV criteria, with demonstrated reliability and external validity.21 At each follow-up assessment, assessors retrospectively rated mood disorder severity using a 1–6 scale (1 = no symptoms, 2–4 = subthreshold DSM-IV symptoms, 5–6 = full threshold DSM-IV criteria), and comorbid diagnoses on a 1–3 scale (1 = no symptoms, 2 = subthreshold, 3 = full threshold). At intake, history of psychotic symptoms was “present” if either the hallucinations or delusions item on the K-SADS-PL screening interview was rated “threshold, definitely present,” or “moderate” or greater (≥4) on the K-MRS. Over follow-up, hallucinations and delusions were rated weekly on the A-LIFE PSR, and ratings of “definite” (3; 2 = probable, 1 = absent) were considered “present.” History of suicidal ideation at intake was considered positive if K-SADS-P depression summary scores reflected past or current suicidal ideation rated “mild” (3) or greater. Over follow-up, suicidal ideation was rated weekly on the A-LIFE PSR using the K-SADS suicidal ideation 1–6 severity scale.23 Mixed episodes over follow-up were operationalized as either a threshold (A-LIFE PSR 5/6) rating for both depression and hypo/mania during the same follow-up week, or a threshold depression or hypo/mania rating and a subthreshold rating on the opposite pole.25 Lithium exposure was tracked weekly over follow-up using the A-LIFE Psychotropic Treatment Schedule. ICCs for A-LIFE syndromal/subsyndromal mood disorders were ≥0.75.
Hospitalization.
A medical history questionnaire documented history of psychiatric hospitalization at intake. Assessors document receipt and timing of psychosocial treatment over follow-up using the A-LIFE Psychosocial Treatment Schedule.
Physical/Sexual abuse.
History of physical and sexual abuse was assessed at intake and each follow-up via the K-SADS-PL Traumatic Events form.
Statistical analysis
As in prior publications, we applied criteria developed by Fusar-Poli et al.11 to develop the RC. To ensure potential for widespread clinical utility of the resultant RC, we selected predictors from extant systematic reviews on risk factors for suicidal behavior in individuals with bipolar disorder that are readily assessed in clinical practice (Table 2). Strict reliance on our own work identifying predictors of suicidal behavior in COBY may have resulted in model overfitting. Risk factors were trichotomized into three timeframes: threshold presence in the past 6 months; lifetime history only; no lifetime history.
| Predictor |
|---|
| Age |
| Sex at birth |
| Socioeconomic status |
| Bipolar disorder subtype |
| Age of mood disorder onset |
| History of suicide attempt |
| History of non-suicidal self-injurious behavior |
| History of physical/sexual abuse |
| History of psychiatric hospitalization |
| Panic disorder |
| Any anxiety disorder |
| Substance use disorder |
| History of psychotic symptoms |
| History of mixed episodes |
| Depressive episode severity/burden |
| Family history of bipolar disorder |
| Family history of suicide attempt |
| Family history of substance use disorder |
Abbreviation: COBY, Course and Outcome of Bipolar Youth.
The RC was trained via boosted multinomial classification trees,29 a useful model for these data because it implicitly incorporates interactions between predictors and has been shown to effectively predict mood episodes in COBY and elsewhere.17,30 Predictions were then calibrated via Platt scaling.31 To avoid overfitting, approximately half of the sample was used to train the RC; the other half was held out and used to independently test the RC. The training sample was further subdivided into five folds so that Platt scaling could be performed via cross-validation (∼40 subjects and >500 observations per fold).32 We designed and implemented an algorithm to optimize balance between randomized sets and folds in suicide attempt rates, with randomization at the participant level (i.e., repeated measurements within-participant were confined to the same fold; see e-Methods 1). In addition to the boosted model, we trained and tested a parametric prediction model using ridge regression for comparison. Shrinkage parameters were determined using methods from Cule and De lorio.33
To test model validity, we assessed discrimination via area under the receiver operating characteristic curve (AUC) on the testing sample, evaluating the predicted one-year suicide attempt risk. Test sensitivity, specificity, and positive predictive value were assessed at a range of risk thresholds. Calibration was tested via Hosmer-Lemeshow test, and by plotting and comparing observed versus predicted suicide attempt risk. Predictor importance was assessed via the Friedman relative influence statistic29 (computed via cross-validation), and the decrement in AUC with removal of that variable from the model (significance of decrements tested via 95% boot-strap percentile interval). Lastly, an experimental reduced model was trained using only the 10 predictors that comprise the top 90% of cross-validated relative influence in the full model. Lastly, we examined whether predictors of suicide attempt risk similarly predicted the lethality of suicide attempts by reformulating the RC model to instead predict the medical lethality scores associated with each suicide attempt; cross-validated relative influence statistics were similarly calculated for each predictor variable.
RESULTS
Rates of Suicide Attempt Over Follow-Up
Over a median of 13.1 years of follow-up, we observed 249 suicide attempts among 106 individuals. Twenty-seven percent of the sample (106/394) had at least one suicide attempt over follow-up, and 14% (54/394) had ≥2 suicide attempts. The estimated median time to first attempt over follow-up was 4 years (range = 0.5–12 years). Among participants with a suicide attempt over follow-up, median rate was approximately one attempt per 7 years.
One-Year Risk Calculator
The participant randomization procedure yielded a training sample with 200 participants with 2751 observations, and a testing sample with 194 participants with 2776 observations. Except for a slightly higher rate of disruptive behavior disorder in the testing sample (63% vs. 51%, p = 0.01), there were no significant between-subsample differences in length of follow-up, number of assessments, duration between assessments, number of suicide attempts, demographic or clinical factors. Overall, the participant randomization procedure yielded highly balanced subsamples.
The test AUC of the full model's one-year risk predictions was 0.82 (0.79, 0.85; Figure 1). Predicted and observed suicide attempt risk were consistent through the range of risk scores and did not significantly differ (Hosmer-Lemeshow χ2 = 10.24, df = 8, p = 0.25), indicating no evidence of miscalibration. Furthermore, average predicted one-year risk in the testing sample (mean = 0.050, median = 0.029) was similar to the observed suicide attempt rate in the testing sample (0.049). Table 3 shows test prediction metrics across a range of predicted risk thresholds; sensitivity and specificity are maximized at 0.73, with a predicted risk threshold of 0.036 (i.e., if predicted risk ≥0.036, predict SA; if predicted risk <0.036, predict no suicide attempt).
| Predicted risk cutoff decision statistics | ||||
|---|---|---|---|---|
| Risk score cutoff | Proportion of sample in risk group | Sensitivity | Specificity | Positive predicted value |
| 0.030 | 0.46 | 0.91 | 0.56 | 0.10 |
| 0.035 | 0.31 | 0.77 | 0.71 | 0.12 |
| 0.040 | 0.23 | 0.67 | 0.79 | 0.14 |
| 0.045 | 0.18 | 0.58 | 0.84 | 0.16 |
| 0.050 | 0.14 | 0.53 | 0.88 | 0.18 |
Of the 21 predictors in the model (Table 4), 10 accounted for >90% of the cross-validated relative influence in the model (direction associated with increased risk noted in parentheses): (1) age of mood disorder onset (younger); (2) non-suicidal self-injury history (presence); (3) current age (older); (4) history of psychosis (present); (5) SES (lower); (6) most severe past 6 months depressive symptoms (i.e., PSR 1–6; higher); (7) history of suicide attempt (present); (8) family history of suicide attempt (present); (9) SUD history (present); and (10) lifetime history of physical/sexual abuse (present). Of note, when parametrically estimating the predictive influence of each level of a trichotomous variable, presence in the past 6 months reliably predicted higher suicide attempt risk than lifetime history.
| Predictor variable | Across-fold average relative influence | AUC decrement if removed from model |
|---|---|---|
| Age of mood disorder onset | 21.45 | 0.03 |
| History of non-suicidal self-injurya | 14.11 | 0.00 |
| Current age | 13.60 | 0.00 |
| History of psychotic symptomsa | 8.94 | 0.01 |
| Socioeconomic statusb | 8.28 | 0.00 |
| Depressive episode severity/burdenc | 6.74 | 0.00 |
| History of suicide attempta | 5.74 | 0.00 |
| Family history of suicide attempt | 5.35 | 0.00 |
| History of substance use disordera | 4.05 | 0.00 |
| Lifetime history of physical/sexual abuse | 3.87 | 0.01 |
| Family history of bipolar disorder | 1.43 | |
| Hospitalization in previous 6 months (y/n) | 1.40 | |
| Duration of threshold depression symptomsd | 1.20 | |
| Threshold suicidal ideation historya | 1.13 | |
| Female (sex at birth) | 1.01 | |
| Panic disorder | 0.54 | |
| Family history of substance use | 0.41 | |
| Any anxiety disorder | 0.39 | |
| Social support over previous 6 months | 0.35 | |
| Bipolar disorder subtype (NOS vs. I/II) | 0.02 | |
| Mixed episode in the previous 6 months (y/n) | 0.00 | |
Note: Bold cells comprise the top 90% of relative influence. Note that no individual predictor's removal from the model yielded a statistically significant AUC decrement after bootstrap.
Abbreviations: AUC, area under curve; COBY, course and outcome of bipolar youth; A-LIFE PSR, adolescent-longitudinal interval follow-up evaluation psychiatric status ratings (Keller et al., 1987).
a
Present during previous 6 months versus lifetime history only versus no lifetime history.
b
Hollingshead-Redlich (1975) criteria.
c
Most severe depression symptoms in the past 6 months (A-LIFE PSR 1–6).
d
6 months of continuous symptoms versus intermittent symptoms versus no symptoms.
When reformulating the model to predict medical lethality of suicide attempts, nine out of the ten most influential predictors in the RC outlined above remained among the top ten most influential predictors of medical lethality (Table S1). The notable exception was non-suicidal self-injury history, which, while highly predictive of risk for suicide attempt, was not highly predictive of medical lethality. This predictor was replaced by family history of bipolar disorder as one of the ten predictors comprising more than 90% of the cross-validated relative influence in the medical lethality model.
Sensitivity analyses indicated no significant AUC decrements after individually eliminating any predictors in the model, with the largest decrement = −0.03 (−0.04, 0.03) when removing age of mood disorder onset. Sensitivity analyses excluding youth with bipolar-II and –NOS (i.e., limited to bipolar-I only) yielded a minimal increase in AUC (0.83), with the same 10 predictors accounting for the top 90% of relative influence in the model. Similarly, sensitivity analyses excluding observations with concurrent lithium use, and separately including only those with concurrent lithium use, showed no significant change in AUC. Substituting dichotomous (i.e., presence in the past 6 months; lifetime history) for trichotomous risk factors in the model yielded nearly identical AUCs and influential variables, indicating minimal impact on model performance. Lastly, dividing substance use disorder history into two separate predictor variables for alcohol use disorder and other substance (i.e., drugs) use disorder yielded a nearly identical AUC. Drug use disorder was a slightly more influential predictor than alcohol use disorder (4.54% relative influence vs. 1.89%). However, it is important to note that drug use disorder over follow-up was slightly more prevalent than alcohol use disorder in this sample (31% of participants vs. 26%), and within participants with substance use disorder, drug use disorder was much more persistent over follow-up than alcohol use disorder (27% of follow-up weeks on average vs. 16%).
The ridge regression trained and tested for comparison to the boosted model performed comparably weaker, with a test AUC = 0.71 and maximized sensitivity and specificity at 0.65 with predicted risk threshold = 0.0375. Predicted and observed suicide attempt risk were consistent through the range of risk scores and did not significantly differ (Hosmer-Lemeshow χ2 = 6.58, df = 8, p = 0.58), indicating no evidence of miscalibration.
A reduced model was trained using only the ten predictors accounting for the top 90% of cross-validated relative influence. Results indicated that model discrimination remained approximately unchanged in the reduced model at 0.82 (0.79, 0.84), and optimal sensitivity and specificity improved to 0.74.
DISCUSSION
We describe the first individual-level 1-year RC for suicide attempt among the high-risk group of individuals with early-onset bipolar disorder. This RC offers promise for clinical use due to its reliance on just ten readily assessed demographic and clinical variables, including current age, age of mood disorder onset, SES, family history of suicide attempt, personal history of suicide attempt, non-suicidal self-injury and psychotic symptoms, and recent (past 6 months) threshold depressive symptoms. Additionally, sensitivity and specificity (AUC = 0.82) of the RC meets or exceeds that of most RCs utilized in clinical practice for other medical (e.g., cardiovascular disease, AUC = 0.76–0.79) and psychiatric conditions (e.g., new-onset psychosis, AUC = 0.71).11
Alternative approaches to prediction of suicidal behavior in high-risk groups include screening tools (e.g., the Comprehensive Suicide Risk Assessment for veterans34) and self-reports (e.g., the Concise Health Risk Tracking Self-Report for adults with BP35), but these have lower sensitivity and specificity for predicting near-term suicide attempt, and rely on patient-reported symptoms which may be subject to bias. Recent studies have attempted to enhance reliability and scalability of suicide risk prediction tools by applying machine learning algorithms to electronic medical record data, for example,36 yielding models with similar predictive power as this RC. While these models have the benefit of automated, large-scale risk detection in healthcare systems, they are limited to individuals for whom prior medical records are available. Thus, models dependent on electronic medical record data from within a single healthcare system may not be reliable or even implementable among individuals with mental health disorders, who are more likely to be uninsured and seek treatment from safety net providers.37 Indeed, when de la Garza and colleagues38 applied machine learning to predict suicide attempt in a representative sample of adults in the Unites States, socioeconomic disadvantage, lower education level and past year financial crisis were among the ten most potent predictors, highlighting inherent behavioral healthcare disparities.
A clinical tool like the RC which we describe relies on demographic and clinical information commonly gathered during standard psychiatric assessment, and as such, may hold promise to enhance clinical care. For example, results could trigger notifications alerting providers of individuals at elevated risk and changes in risk status. Elevated risk scores could further indicate the need for more detailed imminent suicide risk assessment using evidence-based tools, as well as creation, update, and/or review of the individual's safety plan.39 Subsequent intervention may include consideration of appropriate frequency, type and level of care for the individual's predicted risk. Additionally, outreach efforts may be initiated for high-risk patients who do not attend scheduled visits, as implemented in the Veterans Affairs health system.40 This approach is closely aligned with the aims of the American Foundation for Suicide Prevention's “Project 2025,41” a nationwide initiative to reduce the annual suicide rate 20% by 2025. This plan specifically highlights enhanced identification of at-risk individuals via evidence-based practices within systems of primary and behavioral health care as one of four priority areas with greatest potential for impact.
While the RC we describe yields highly accurate individual-level prediction, a RC score does not replace clinical judgment. Indeed, additional factors shown to predict elevated risk for suicidal behavior among youth with bipolar disorder—including several included herein—did not emerge as highly influential in the RC (e.g., sex at birth, mixed symptoms25), it is possible that such factors exert their effects through the more highly predictive factors in the RC. These and other established risk factors (e.g., comorbid personality disorder5) should also be considered for select individuals with early-onset bipolar disorder. In some cases, the RC may under- or overestimate risk, and additionally does not yield detailed information on specific aspects of suicide risk (e.g., intent, lethality), rendering further clinical and suicide risk assessment critical. Of note, our analyses examining a secondary RC to predict suicide attempt lethality demonstrates remarkable predictor overlap with the RC to predict suicide attempt, with nine of the top ten most influential variables remaining the same. The exception is history of non-suicidal self-injury, which is highly predictive of risk for suicide attempt, but not of medical lethality. Together, these findings indicate the RC holds promise as a data-driven tool to augment and guide suicide risk assessment and treatment planning.
Regarding research implications, the RC may help identify individuals during elevated periods of risk for whom novel suicide prevention approaches may be examined. Additionally, future studies may further enhance person-level prediction of near-term suicide risk by combining the RC with cognitive tasks and/or biological indices (e.g., neuroimaging, polygenic risk scores) implicated in suicide risk.10
Strengths of this study include the well-characterized sample of individuals with early-onset bipolar disorder followed longitudinally with validated measures. However, included risk factors were limited to those collected and, therefore, are not exhaustive (e.g., emotional abuse). Additional limitations include a primarily White sample, rendering examination among more diverse samples a critical future direction. COBY participants were recruited from clinical settings, possibly limiting generalizability of the results. Next steps include external validation in an independent sample of individuals with early-onset bipolar disorder, and examination among community samples and individuals with adult-onset bipolar disorder. Future work may also aim to identify models that reliably predict risk over even more proximal timeframes (e.g., 1 month).
Using just ten demographic and clinical factors commonly assessed in clinical practice, this individual-level RC for early-onset bipolar disorder reliably predicts one-year suicide attempt risk. This low-burden tool, if validated, offers promise to enhance suicide risk assessment and management in this high-risk population.
Acknowledgments
ACKNOWLEDGEMENTS
The authors wish to thank the youth and their family members who participated in the Course and Outcome of Bipolar Youth (COBY) study the COBY research team, and administrative support from Pamala Pyle. This research was supported by National Institute of Mental Health (NIMH) Grants MH59929 (PI: Boris Birmaher, MD), MH59691 (PIs: Martin B. Keller, MD, Shirley Yen, PhD), and MH59977 (PI: Michael Strober, PhD).
Footnotes
DATA AVAILABILITY STATEMENT
The investigative team contributes data and associated documentation in compliance with NIMH Data Sharing Policy through the National Database for Clinical Trials Related to Mental Illness (NDCT) as per NOT-MH-14-015. Per the NIMH Data Repositories Data Access Agreement and Use Certification, in order to gain access to data from this project, outside Investigators must submit a detailed proposal of the study aims, hypotheses to be tested, variables/constructs and analytic approach to be used.
ORCID
Tina R. Goldstein https://orcid.org/0000-0002-4762-8060
Danella Hafeman https://orcid.org/0000-0001-8312-3513
Heather Hower https://orcid.org/0000-0002-9411-2059
Lauren Weinstock https://orcid.org/0000-0001-9970-235X
Boris Birmaher https://orcid.org/0000-0001-9299-6519
REFERENCES
1.
Curtin SC, Heron M. Death Rates Due to Suicide and Homicide among Persons Aged 10–24: United States, 2000–2017. Vol. 352. 2019. NCHS Data Brief. https://www.cdc.gov/nchs/products/index.htm
2.
Goldstein TR. Suicidality in pediatric bipolar disorder. Child Adolesc Psychiatr Clin N Am. 2009;18(2):339-352.
3.
Simon G, Hunkeler E, Firmena B, Lee J, Savarino J. Risk of suicide attempt and suicide death in patients treated for bipolar disorder. Bipolar Disord. 2007;9:526-530. doi:
4.
Dome P, Rohmer Z, Gonda X. Suicide risk in bipolar disorder: a brief review. Medicina. 2019;55(8):403. doi:
5.
Schaffer A, Isometsa E, Tondo L, et al. International society for bipolar disorders task force on suicide: meta-analyses and meta-regression of correlates of suicide attempts and suicide deaths in bipolar disorder. Bipolar Disord. 2015;17(1):1-16. doi:
6.
da Silva CL, Alencar AP, Neto PJN, et al. Risk factors of suicide in bipolar disorder: a systematic review. J Affect Disord. 2015;170:237-254. doi:
7.
Tidemalm D, Haglund A, Karanti A, Landen M, Runeson B. Attempted suicide in bipolar disorder: risk factors in a cohort of 6086 patients. PLoS One. 2014;9(4):e9409. doi:
8.
Franklin JC, Ribeiro JD, Fox KR, et al. Risk factors for suicidal thoughts and behaviors: a meta-analysis of 50 years of research. Psychol Bull. 2017;143(2):187-232. doi:
9.
Tran T, Luo W, Phung D, et al. Risk stratification using data from electronic medical records better predicts suicide risks than clinician assessments. BMC Psychiatry. 2014;14:76.
10.
Roos L, Sareen J, Bolton JM. Suicide risk assessment tools, predicitve validity findings and utility today: time for a revamp? Neuropsychiatry. 2013;3(5):1-13.
11.
Fusar-Poli P, Rutigliano G, Stahl D, et al. Development and validation of a clinically based risk calculator for the transdiagnostic prediction of psychosis. JAMA Psychiat. 2017;74(5):493-500. doi:
12.
Nobel L, Mayo NE, Hanley J, Nadeau L, Daskalopoulou SS. MyRisk_ Stroke calculator: a personalized stroke risk assessment tool for the general population. J Clin Neurol. 2014;10(1):1-9. doi:
13.
Gail MH, Brinton LA, Byar DP, et al. Projecting individualized probabilities of developing breast cancer for white females who are being evaluated annually. J Natl Cancer Inst. 1989;81(24):1879-1886. doi:
14.
Perlis RH. A clinical risk stratification tool for predicting treatment resistance in major depressive disorder. Biol Psychiatry. 2013;74(1):7-14. doi:
15.
Hafeman D, Merranko J, Goldstein TR, et al. Assessment of a person-level risk calculator to predict new-onset bipolar spectrum disorder in youth at familial risk. JAMA Psychiat. 2017;74(8):841-847. doi:
16.
Birmaher B, Merranko J, Goldstein TR, et al. A risk calculator to predict the individual risk of conversion from subthreshold bipolar symptoms to bipolar disorder I or II in youth. J Am Acad Child Adolesc Psychiatry. 2018;57(10):755-763. doi:
17.
Birmaher B, Merranko JA, Gill MK, et al. Predicting personalized risk of mood recurrences in youths and young adults with bipolar spectrum disorder. J Am Acad Child Adolesc Psychiatry. 2020;59(10):1156-1164. doi:
18.
Spittal MJ, Pirkis J, Miller M, Carter G, Studdert DM. The repeated episodes of self-harm (RESH) score: a tool for predicting risk of future episodes of self-harm by hospital patients. J Affect Disord. 2014;161:36-42. doi:
19.
Fazel S, Wolf A, Larsson H, Mallett S, Fanshawe TR. The prediction of suicide in severe mental illness: development and validation of a clinical prediction rule (OxMIS). Transl Psychiatry. 2019;9:98. doi:
20.
Melhem NM, Porta G, Oquendo M, et al. Severity and variability of depression symptoms predicting suicide attempt in high-risk individuals. JAMA Psychiat. 2019;76(6):603-612.
21.
Birmaher B, Axelson D, Strober M, et al. Clinical course of children and adolescents with bipolar spectrum disorders. Arch Gen Psychiatry. 2006;63(2):175-183. doi:
22.
Kaufman J, Birmaher B, Brent D, et al. Schedule for affective disorders and schizophrenia for school-age children-present and lifetime version (K-SADS-PL): initial reliability and validity data. J Am Acad Child Adolesc Psychiatry. 1997;36(7):980-988. doi:
23.
Chambers WJ, Puig-Antich J, Hirsch M, et al. The assessment of affective disorders in children and adolescents by semistructured interview: test-retest reliability of the schedule for affective disorders and schizophrenia for school-age children, present episode version. Arch Gen Psychiatry. 1985;42(7):696-702. doi:
24.
Axelson DA, Birmaher B, Brent D, et al. A preliminary study of the kiddie schedule for affective disorders and schizophrenia for school-age children mania rating scale for children and adolescents. J Child Adolesc Psychopharmacol. 2003;13(4):463-470. doi:
25.
Goldstein TR, Ha W, Axelson DA, et al. Predictors of prospectively observed suicide attempts among youth with bipolar disorder. Arch Gen Psychiatry. 2012;69(11):1113-1122. doi:
26.
Hollingshead A. Four-Factor Index of Social Status. Yale University; 1975.
27.
First MB, Gibbon M. The structured clinical interview for DSM-IV Axis I disorders (SCID-I) and the structured clinical interview for DSM-IV Axis II disorders (SCID-II). In: Hilsenroth MJ, Segal DL, eds. Comprehensive Handbook of Psychological Assessment: Personality Assessment. John Wiley & Sons, Inc.; 2004:134-143.
28.
Weissman MM, Wickramaratne P, Adams P, Wolk S, Verdeli H, Olfson M. Brief screening for family psychiatric history: the family history screen. Arch Gen Psychiatry. 2000;57(7):675-682. doi:
29.
Friedman JH. Greedy function approximation: a gradient boosting machine. Ann Stat. 2001;29(5):1189-1232.
30.
Van Voorhees BW, Paunesku D, Glollan J, Kuwabara S, Reinecke M, Basu A. Predicting future risk of depressive episode in adolescents: the Chicago adolescent depression risk assessment (CADRA). Ann Fam Med. 2008;6(6):503-511. doi:
31.
Niculescu-Mizil A, Caruana R. Obtaining Calibrated Probabilities from Boosting. Association for Uncertainty in Artificial Intelligence Press; 2005:413-420.
32.
Platt JA. Probabilities for SV machines. In: Smola JA, Bartlett PL, Scholkopf B, Schuurmans D, eds. Advances in Large Margin Classifiers. MIT Press; 2000:61-74.
33.
Cule E, De Iorio M. Ridge regression in prediction problems: automatic choice of the ridge parameter. Genet Epidemiol. 2013;37(7):704-714. doi:
34.
Doran N, De Peralta S, Depp C, et al. The validity of a brief risk assessment tool for predicting suicidal behavior in veterans utilizing VHA mental health care. Suicide Life Threat Behav. 2016;46(4):471-485. doi:
35.
Reilly-Harrington NA, Shelton RC, Kamali M, et al. A tool to predict suicidal ideation and behavior in bipolar disorder: the concise health risk tracking self-report. J Affect Disord. 2016;192:212-218. doi:
36.
Simon GE, Johnson E, Lawrence JM, et al. Predicting suicide attempts and suicide deaths following outpatient visits using electronic health records. Am J Psychiatry. 2018;175(10):951-961. doi:
37.
Mechanic D, Olfson M. The relevance of affordable care act for improving mental health care. Annu Rev Clin Psychol. 2016;12:515-542.
38.
de la Garza A, Blanco C, Olfson M, Wall M. Identification of suicide attempt risk factors in a national US survey using machine learning. JAMA Psychiat. 2021;78(4):398-406.
39.
Stanley B, Brown G. Safety planning intervention: a brief intervention to mitigate suicide risk. Cogn Behav Pract. 2012;19(2):256-264. doi:
40.
McCarthy JF, Bossarte RM, Katz IR, et al. Predicitve modeling and concentration of the risk of suicide: implications for preventive interventions in the US Department of veterans affairs. Am J Public Health. 2015;105(9):1935-1942.
41.
Prevention AFfS. A nationwide initiative to reduce the annual rate of suicide in the US 20 percent by 2025. 2017. Accessed 12/3/21. https://project2025.afsp.org/
Information & Authors
Information
Published In
History
Published in print: Fall 2023
Published online: 16 October 2023
Keywords
Authors
Competing Interests
CONFLICT OF INTEREST
Competing Interests
Dr. T. Goldstein reports grants from NIMH, The American Foundation for Suicide Prevention (AFSP), University of Pittsburgh Clinical and Translational Science Institute (CTSI), and The Brain and Behavior Foundation and royalties from Guilford Press, outside the submitted work. Dr. Hafeman reports grants from NIMH and the Brain and Behavior Research Foundation. Dr. Weinstock reports research support from NIMH, NCCIH, the NIH Office of Behavioral Social Sciences Research (OBSSR), the Warren Alpert Foundation and the National Institute of Justice (NIJ). Dr. Yen reports research support from NIMH, the National Center for Complimentary and Integrative Health (NCCIH), and AFSP. Dr. B. Goldstein reports grant funding from Brain Canada, Canadian Institutes of Health Research, Heart & Stroke Foundation, NIMH, and the departments of psychiatry at the University of Toronto, and acknowledges salary support from the RBC Investments Chair, held at the Centre for Addiction and Mental Health and the University of Toronto department of psychiatry. Dr. Keller reports research support from NIMH and the John J. McDonnell and Margaret T. O'Brien Foundation. Dr. Strober reports research support from NIMH, and support from the Resnick Endowed Chair in Eating Disorders. Dr. Ryan reports grants from NIH and Axsome Therapeutics. Dr. Birmaher reports grants from NIMH, royalties from Random House, UpToDate and Lippincott, Williams & Wilkins. Mr. Merranko, Ms. Gill, Ms. Liao, Mr. Sewall and Ms. Hower report no financial relationships with commercial interests.
Funding Information
FUNDING INFORMATIONNational Institute of Mental Health, Grant/Award Number: MH059929, MH59691 and MH59977
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).