Trainer Fidelity as a Predictor of Crisis Counselors’ Behaviors With Callers Who Express Suicidal Thoughts

Only posttraining outcome data were included in the analyses because outcome data that preceded the ASIST training was not relevant to linking outcomes and fidelity. We first aligned the silent-monitored outcome variables from 764 calls with the relevant segments of the training and the fidelity ratings of the trainer who trained counselors in the center. For example, the segment of the training that focuses on engaging suicidal individuals in a discussion about reasons for living corresponds to a silent-monitored outcome variable. Each segment yielded trainer fidelity scores (adherence and competence) that were examined in terms of associated outcome variables. Four of the seven segments produced significant results: asking about suicide, contracting a safeplan, process of intervention, and bridge simulation. As the final segment of ASIST, bridge simulation is a highly interactive role-playing technique that incorporates all of the knowledge and skills imparted by the trainers. Therefore, we examined all of the outcome variables with this segment’s fidelity scores.

The RCT protocol was approved by the Institutional Review Board (IRB) of the New York State Psychiatric Institute and Columbia University. This study was approved by the University of Rochester IRB.

The generalized estimating equation (GEE) analysis was employed to address clustered data resulting from the nested study design (counselors nested within centers) and because it is considered preferable to a parametric alternative such as a generalized linear mixed-effects model (10). Variability in number of calls received by counselors was addressed by weighting each center by using the reciprocal of the number of calls from the center. The effect of trainer adherence with and competence in specific segments on counselor behaviors during the calls (for example, asking about current suicidal thoughts and safeplan elements), positive counselor behaviors, negative counselor behaviors, and a global rating of overall effectiveness were examined. The identity, logit, or log link was used depending on whether the outcome was treated as a continuous, binary, or count response. If a categorical outcome had more than two categories, we fit the multinomial response model or the proportional odds model, the latter of which is a simplified version of the multinomial response model, if assessment by goodness-of-fit tests indicated that the latter model was more appropriate (10). Some outcomes had a large percentage of 0 scores (for example, 80% of outcomes for negative counselor behaviors were rated 0, or “not at all”); the hurdle model was used to fit such outcomes. The hurdle model has two components: the logistic component models the effect of adherence and competence on the probability of whether counselors have negative behaviors (1 versus 0), and the Poisson component models the effect of adherence and competence on the variability of negative behaviors (number of negative behaviors) for the subgroup of counselors with recorded negative behavior. For binary and categorical responses, the lowest level was set as the reference level.

All statistical tests were two sided; p values of <.05 were considered statistically significant. All statistical analyses used SAS, version 9.3.

Table 1 presents results for all segments. GEE estimates (betas) show that 12 of the 16 significant findings are for ASIST trainer competence. For example, higher observed trainer competence in the asking about suicide segment was associated with greater instances of counselors asking about prior suicidal thoughts and current plans. ASIST trainer competence during the bridge simulation segment was also associated with counselors reminding callers of past survival skills and identifying personal and formal resources. Greater trainer adherence during the bridge simulation was positively associated with the counselor asking about or exploring suicidal actions with callers and exploring ways for callers to manage psychological pain. Two significant findings for adherence during the bridge simulation were not in the expected direction: the more adherent the ASIST trainer, the less likely counselors were to explore prior suicide attempts, and the rare instances of negative counselor behaviors were associated with greater trainer adherence.

This is the first study to examine the relationship between trainer fidelity and suicide prevention program outcomes among crisis hotline counselors in the context of a TTT model. Overall, we found that trainer competence, more than trainer adherence, was associated with better performance by crisis line counselors with callers who expressed thoughts of suicide. Greater trainer adherence to program content was actually predictive of poorer outcomes in two cases.

How do we account for the finding that program content had little impact on counselor behaviors? One factor may be that these counselors were already working on the crisis line and had prior experience in suicide prevention. They, like experienced clinicians who resist learning new interventions (11), may not have been persuaded to adopt new information or practices. Moreover, consistent with previous findings that strict adherence to an intervention’s manual may be associated with poorer therapeutic outcomes (12), adherence to the ASIST program had a deleterious effect on some counselor outcomes. Future studies should study outcomes for novice counselors without prior experience or training. The reason for the positive impact of trainer competence on counselor outcomes is not known; however, it is consistent with the finding that therapist competence (but not adherence) predicted positive outcomes among adolescents in substance abuse treatment (13). Future studies on the relative importance of competence and adherence in a variety of training contexts are needed to address this gap.

Regarding limitations of the study, we did not have individual counselor or trainer data, such as the amount, type, or duration of previous crisis training. Nor did we have data on job duration or status (volunteer or paid) or center culture or climate. These factors may moderate the relationship between trainer fidelity and outcomes. We also did not have information about modifications (14) that the trainers may have made to the ASIST program.

Trainer competence was positively associated with counselors’ use of detailed risk assessment (asking about suicide thoughts and plans) and safety planning (identifying resources) with callers who expressed suicidal thoughts. ASIST trainer delivery is more important than program content when the “end users” are suicide crisis line counselors. Our results are consistent with findings by Gould and others (4) that ASIST-trained counselors were not significantly more likely than counselors who had not received ASIST to ask about or explore a caller’s current suicide plans, preparatory behaviors intent, or prior suicide thoughts or attempts. In this study, adherence to ASIST per se, as opposed to trainer competence, was not associated with more comprehensive risk assessments. The implication of this finding is that programs should prioritize the selection of trainers who have group facilitation skills, are comfortable directing experiential learning activities, and are willing to embrace new program content.