Effects of Training on Suicide Risk Assessment

Participants in the risk assessment training group consisted of 45 trainees who attended a workshop on risk assessment on either of two occasions on which it was offered to clinicians in a university-based psychiatry department. Forty-three participants were residents in psychiatry (15 were in the first postgraduate year, 22 were in the second postgraduate year, and six in the third postgraduate year), and two were clinical psychology interns.

The comparison group participants were ten psychiatric residents who were in their first postgraduate year and attended a three-hour workshop on the application of evidence-based medicine to psychiatry. This workshop was not focused specifically on risk assessment for suicide.

Data were collected between July 2004 and July 2007. The protocol was approved by the Committee on Human Research of the University of California, San Francisco. Participants gave informed consent to participate in the anonymous evaluation of the training.

The risk assessment training consisted of a five-hour workshop on evidence-based assessment and management of risk of suicide and violence. This brief report describes the component concerning suicide. (See reference 13 for a description of the training in risk assessment for violence.) The workshop was led by 12 faculty psychiatrists and psychologists, including clinical leaders from various clinical services in the psychiatry department.

Participants in the risk assessment training group received a lecture on assessment and management of suicide risk based on the American Psychiatric Association Practice Guidelines for the Assessment and Treatment of Patients With Suicidal Behavior ( 10 ) and relevant materials. Drawing upon the literature on structured professional judgment ( 14 ), a model was presented that organizes risk factors for suicide according to those that are historical (past), are clinical (present), and involve risk management (future). This approach applies clinical guidelines to assist decision making by prompting the clinician to consider information about core risk and protective factors that the literature suggests need to be taken into account in a reasonably comprehensive risk assessment. Based on consideration of these risk and protective factors, participants were taught how to use judgment in determining whether the risk is low, moderate, or high, as well as how to develop a risk management plan based on the level of assessed risk.

Participants then took part in small groups of four to eight attendees that were facilitated by faculty members. The small groups reviewed two case vignettes (which were different from the vignettes given for study purposes), and they were asked to identify historical, clinical, and risk management factors in each vignette and to envision scenarios that would affect the patients' level of risk of suicide. They were also asked to estimate the patients' level of risk and to develop intervention plans appropriate to the level of risk.

Participants then received a lecture on medical-legal aspects of documentation of risk assessment. The lecture recommended gathering information about risk factors and protective factors, rationally weighing the significance of those factors to estimate the level of suicide risk, developing and implementing a plan of intervention to reduce the risk, and documenting the process ( 4, 6, 11 ).

Before and after the workshops, participants in the risk assessment training group and in the comparison group rated items on a 7-point scale concerning their perceived ability to accurately assess patients' risk of suicide, their ability to manage patients' risk of suicide, and their knowledge about suicide and working with suicidal patients. In addition, before and after each workshop, participants in both groups wrote a progress note in response to a case vignette that included the assessment and plan regarding the patient's imminent risk of suicide. The order of the two vignettes was counterbalanced, so that about half of the participants responded to a vignette about one patient before the workshop and the other vignette about another patient afterward; the other half of the participants received the vignettes in reverse order.

The progress notes were rated independently by two clinicians (a senior psychiatrist and one of two advanced clinical psychology fellows) who did not know whether the notes had been written before or after participants attended the workshops. The ratings had two main aspects. First, the raters applied a structured content analysis that included 19 variables that were based on the literature on standard of care in risk assessment for suicide ( 4, 6, 11 ) and on dimensions developed in an earlier pilot version of the training. These ratings included whether participants identified specific risk and protective factors for suicide, whether participants explained their thought process in determining the risk level (that is, rational weighing of the risk factors and protective factors in developing a risk assessment and intervention plan), and organization of thinking about risk management (for example, why specific risk management plans were selected and alternatives were rejected). Each item was coded as 0, item absent; 1, item possibly present; or 2, item present. We calculated a composite score consisting of the sum of the individual items for purposes of data analysis. The raters also coded the overall quality of the documentation concerning risk assessment for suicide on a scale ranging from 1, poor, to 7, excellent.

Interrater reliability, based on the average intraclass correlation coefficient, was r=.93 for the composite score of the ratings of 19 indicators of documentation quality and r=.75 for the global rating of the quality of the progress note. The average of the ratings of the two coders was used for data analysis.

To evaluate the extent to which participation in risk assessment training accounted for changes in documentation, we used multiple regression analysis, with models that included the pretest score, a dummy variable indicating the order of presentation of the vignettes, and a variable indicating whether the participant had received the training in risk assessment for suicide. In each case, the dependent variable was a change score that measured the degree to which the posttest score differed from the pretest score. We used Cohen's ƒ ² to determine the effect size of adding the variable representing whether participants received risk assessment training to the models that included only the pretest score and order of presentation of the vignettes ( 15 ). Conventionally, ƒ ² values of .02, .15, and .35, respectively, are considered to represent small, medium, and large effect sizes. We used a similar strategy to determine the extent to which risk assessment training accounted for changes in self-rated competence. We analyzed data with SPSS, version 15.0.

The participants in the risk assessment training group did not differ significantly from those in the comparison group in mean±SD hours of prior formal training in assessing and managing suicide risk (4.10±4.7 hours for the risk assessment training group and 1.6±1.4 hours for the comparison group) (t =-1.64, df=52, p=.11). Participants in the risk assessment training group did, however, report significantly more years of previous experience in providing mental health services, compared with the comparison group (2.0±2.1 years versus .6±.6 years) (t=-2.17, df=53, p<.04).

Table 1 shows that there was a significant difference between the two groups, with participants in the risk assessment training group showing a larger change score, which indicated greater improvement in their ability to identify risk and protective factors for suicide and in their ability to articulate better-organized reasoning about risk assessment and risk management strategies ( β =.393, p=.001). The size of this effect was medium ( ƒ ² =.26).

Table 1 also shows that participants in the risk assessment training group demonstrated significantly more improvement in the overall quality of their documentation concerning suicide risk assessment, compared with those who did not receive the training ( β =.340, p=.001). The size of the effect was medium ( ƒ ² =.22).

The control variables (that is, order of presentation of the vignettes and the pretest score) also predicted the outcome variables. Vignette order accounted for variability in the stimuli used to measure skill in suicide risk assessment. The pretest score measured baseline skill in risk assessment. Participants with lower baseline skill showed greater change on the outcome variables.

As shown in Table 2, multiple regression analyses revealed statistically significant and large effect sizes for the effect of participation in the risk assessment training on increases in self-ratings of ability to assess patients' risk of suicide ( ƒ ² =.47), knowledge about suicide and working with suicidal patients ( ƒ ² =.44), and ability to manage patients' risk of suicide ( ƒ ² =.44). In addition, participants who rated their competence in suicide risk assessment as lower at pretest showed greater increases in self-rated skills.

These results show that systematic training in evidence-based suicide risk assessment can improve clinical documentation. Progress notes composed after training showed that participants were able to identify in a more systematic manner historical, clinical, and risk management variables that pertain to suicide risk, as well as consider in a more explicit manner the significance of these risk and protective factors when developing interventions to reduce patients' risk of suicide. Clinicians could better communicate the reasons for their risk assessments and risk management plans after receiving specific training in this subject. The improvements in documentation shown by participants in the risk assessment training group were substantial, compared with those of participants in a more general workshop on evidence-based practice, and these improvements held up in analyses that controlled for the participants' baseline skill. Overall, these findings support the conclusion that training in evidence-based suicide risk assessment can improve case formulations, so that they are more explicit and consistent with the scientific and professional literature.

Risk assessment training also was associated with increased self-confidence in risk assessment skill. Although self-ratings may be vulnerable to demand characteristics associated with the context of evaluating the workshop, it is of interest that participants in the risk assessment training group showed increases in self-rated competence that paralleled the objective improvements in suicide risk assessment.

Further research, with larger samples, would be useful to determine the extent to which the acquired skills in risk assessment are retained over time. In addition, future research could assess the extent to which clinical progress notes with actual patients show improvements similar to those that we observed in response to the case vignettes. However, investigation of that issue would be complicated by the fact that individual patients vary in the extent to which they have risk and protective factors for suicide. Our study controlled for this source of variability by using standard case vignettes, permitting interpretation of changes in documentation as being due to the effect of training.

This research was supported in part by grant 21-MH-18261 from the National Institute of Mental Health.

Dr. Chamberlain is on speakers bureaus for Bristol-Myers Squibb, Wyeth Pharmaceuticals, and Pfizer Pharmaceuticals. The other authors report no competing interests.

The authors are affiliated with the Department of Psychiatry, University of California, San Francisco, 401 Parnassus Ave., San Francisco, CA 94143-0984 (e-mail: [email protected]). Preliminary results were presented at the conference of the American Psychology-Law Society, Division 41 of the American Psychological Association, March 5–8, 2008, Jacksonville, Florida.