The concept of recovery has historical and social underpinnings in the consumer movement in health care and in civil and human rights (
1,
2). Recovery is the subject of a transgressional discourse that may disrupt the dominant biomedical model in mental health settings. Recovery emphasizes the centrality of self-determination, autonomy, collaborative relationships, hope and optimism, a positive preferred sense of identity, meaning in daily life despite the presence of mental illness, and empowerment and personal responsibility (
3–
5).
Recovery-oriented practice underpins mental health service delivery and policy worldwide (
6–
9). In Australia, the National Framework for Recovery-Oriented Mental Health Services (
10) guides the development of service cultures that promote well-being, resilience, and self-determination of people with lived experience, together with their families, carers (equivalent to “caregivers” in U.S. health care contexts, used hereafter), and others providing support. Evidence supporting recovery-oriented practice in mental health settings indicates the effectiveness of interventions to improve employment outcomes, empowerment, peer support, and self-management (
5,
11–
13).
Even though the concept of recovery-oriented practice is widely accepted in policy making, challenges in authentic implementation remain (
14). Keys to successful implementation include a lived-experience (consumer and caregiver) workforce, changes in culture, strong leadership, workforce training, values-based practice, and positive staff attitudes toward people’s ability to recover (
14–
16).
The collaborative recovery model (CRM) is a person-centered, strengths-based, well-being–oriented, and future-focused practice model that involves coaching and has applications for individual- and family-focused recovery interventions and in organizational and workforce development (
17).
CRM training was evaluated across Australian clinical and nonclinical mental health settings in a 5-year, multisite study (
16,
18). The training was shown to improve mental health workers’ knowledge, attitudes, and hopefulness for recovery in community-based clinical and nonclinical mental health settings (
19). This finding was replicated and extended to include therapeutic optimism (
20). No single measure of CRM training has been established as the standard for evaluation. Barriers to evaluating recovery training may include the time needed for evaluation, low response rates, outdated language related to the concept of recovery, and the time required for appropriate selection of an evaluation measure (
16,
21). Research has yet to measure changes in staff skills, confidence, perceived importance of CRM, and language development in relation to CRM training programs.
In this study, we selected CRM because it was developed for Australian clinical mental health services and is applicable across the life span (i.e., including children, youths, families, adults, and older adults) to enable recovery-oriented practice with consumers and their families, caregivers, and other support people, as well as organizational, cultural, and workforce development and sustainable culture change (
17,
22). The recovery implementation steering committee of the mental health program of Eastern Health, Australia, chose CRM as part of an organization-wide recovery-oriented transformation. People with lived experience of mental health recovery (including caregivers) were integral at all levels: in the steering committee, development of training, implementation, delivery, and evaluation. This CRM implementation was the largest undertaken by a single organization to date.
This study aimed to evaluate the effectiveness of the initial 3-day CRM training program and booster training session on mental health staff’s knowledge, attitudes, and skills related to recovery-oriented practice. This study was guided by the following research questions: What difference did CRM training make to staff’s self-rating of knowledge, attitudes, and skills; importance of implementing CRM; and confidence in individual ability and the organization’s ability to implement CRM? After booster training, were changes sustained in staff knowledge, attitudes, skills, importance of implementation, and confidence in implementing CRM? How did CRM training influence the language staff used to define recovery?
Methods
Study Design and Administration
Approval from the Eastern Health Human Research and Ethics Committee was obtained for this prospective interventional study (LR94-2017). Study participation was voluntary. All measures were deidentified and stored separately from data.
Intervention
This study evaluated a 3-day CRM training package that was enhanced through coproduction, cofacilitation, and incorporation of coaching interventions at the individual, team, and organizational levels (
23). Implementation in clinical practice was supported by coaching in team-based reflective practice (
16) through use of action learning sets (
24) (see the
online supplement to this article), a network of recovery change leaders, and booster training sessions (
25) provided at 6 months postbaseline.
CRM.
With a focus on well-being and resilience, CRM has two guiding principles: recovery as an individual process and collaboration and autonomy support. The model has four components: change enhancement, collaborative strengths and values identification, collaborative visioning and goal striving, and collaborative action planning and monitoring (
22).
Cofacilitation and coproduction.
To enhance organizational transformation, all training sessions were cofacilitated, and we sought to honor coproduction (coplanning, codesign, codelivery, and coevaluation) (
26,
27) in all CRM planning, training, implementation, and evaluation activities within organizational and resource constraints. We acknowledge that power imbalances were present in training team structures and employment conditions. Coplanning and codesign occurred in the development of CRM in the early 2000s in a university setting and included people with lived experience. Coplanning of organizational implementation occurred when the program purchased the CRM package. At Eastern Health, CRM trainers with clinical and lived experience collaboratively contextualized the CRM training package for a clinical setting and codesigned a new visual representation of the model (see the
online supplement), posters, a brochure, competency domains, research evidence, stories about lived experience (audio and video), an implementation guide, and published-evidence and lived-evidence (i.e., testimonies) reading guides. Stories were collaboratively developed to illustrate lived experience of CRM in order to enhance staff understanding and to demonstrate hope, the development of a preferred identity, pursuit of meaning and purpose, empowerment to take ownership of one’s life (
28), and enactment of strengths and values as resources for recovery, resilience, well-being (
29), and goal striving (
30). Every training session was cofacilitated and coevaluated within the CRM workforce development team.
Interdisciplinary training and workforce development team.
A CRM workforce development team was established, consisting of trainers with various backgrounds: an occupational therapist, social worker, psychiatrist, trainers with lived experience of recovery or caring, and clinical nurse educators. CRM trainers were credentialed through the train-the-trainer process and a local competency assessment (see the online supplement).
The process for CRM trainer credentialing included attendance at an initial training course (5 days), train-the-trainer program (3 days), practice facilitation (3 days), a booster training session at 6 months (1 day), and then annually (1 day) thereafter; trainers completed a generic competency assessment at booster training. CRM coaching competency for trainers was achieved by implementing coaching pairs and the completion of a CRM-specific coaching competency assessment (see the online supplement). Competency was required in each guiding principle and component. Trainers were selected on the basis of the areas in which they first achieved competency.
CRM training schedule.
Training was delivered to the entire mental health workforce, including all clinical, medical, and leadership programs over 20 months (2017–2018). All participants (N=729) were required to attend an initial 3-day CRM training program and booster training sessions at 6 months and then annually. Training was provided to individual teams within service areas to ensure relevance to the local context and to maximize culture change. Training was scheduled to meet service needs and to minimize disruption to service delivery. This method of scheduling training delivery parallels the guiding principles of CRM to “meet people where they are at” and enhance motivation for culture change.
Setting
The study was set in a large metropolitan mental health program servicing a population of 650,000 people living in a geographical area of 2,816 square kilometers. The mental health program comprised 25 sites, including mental health services for infants, children, youths, adults, and older persons, and spanned 13 community care teams and six inpatient units, assertive outreach and crisis assessment teams, consultation-liaison services, subacute and residential services, and a diverse range of specialist programs.
Sample
The sample included all mental health program staff in substantive employment. Staff employed in clinical, medical, and peer worker roles across the mental health program participated, including psychiatrists, other medical staff, nurses, occupational therapists, social workers, psychologists, neuropsychologists, speech pathologists, lived-experience workforce, and senior leadership and service development positions. All mental health program staff who completed CRM training and at least one pre-post training measure were included in this study.
Measures
A CRM pre-post training measure (see the
online supplement) was completed for the initial training sessions at days 0 and 3 and at the 6- to 12-month booster sessions. The measure was developed for this study to assess staff members’ knowledge, attitudes, skills, confidence, and language related to CRM and their perception of the importance of CRM implementation. Likert scales to assess attitude and skills were adapted from Crowe et al. (
19). An initial pilot test (N=40 staff members) identified redundant items. The final test included 10 items to assess attitude and seven items to measure skills, with ratings ranging from 1, strongly disagree, to 5, strongly agree. Scales used to measure attitudes and skills had internal consistency indicated by Cronbach’s α=0.82 and 0.83, respectively. Knowledge was measured by using 10 true-or-false items (score range 0–10, with higher scores indicating greater knowledge). Perceived importance of CRM implementation, confidence in implementing CRM, and confidence in the organization were measured with one item each on a visual analog scale (range 0%–100%, with higher scores indicating greater importance or confidence).
Data Collection
Initial 3-day CRM training was completed by the 729 participants between March 2017 and October 2018. A total of 985 measures were collected: 591 participants completed pretraining measures, 354 posttraining measures, and 40 booster training measures. All measures were deidentified. No inpatient staff completed posttraining or booster training measures.
Data Analyses
Data were analyzed with SPSS, version 28.0 (
31). Participants with complete data were used for the subanalysis of each item.
We could match pretraining, posttraining, and booster training measures by using unique identifiers provided by participants. Not all repeated measures could be matched. In total, 277 participants had at least one matched pre- and posttraining measure, and 36 participants had at least one matched pretraining, posttraining, and booster training result. Missing data were addressed with listwise deletion.
The Kolmogorov-Smirnov statistic showed that normality of the data could not be assumed; nonparametric analyses were therefore undertaken. The Wilcoxon signed rank test was used to compare variables pre- and posttraining, and Friedman tests were used to compare pre- and posttraining and booster training measures.
Staff definitions of recovery were analyzed for content by using word frequency as an indicator of changes in staff’s understanding of recovery and of the development of a shared language. Definitions were entered into analysis software (
worditout.com). The 100 most frequently used words were illustrated in a word cloud, in which the size of the word correlated with the frequency of use (i.e., the larger the depiction of the word, the more frequently the word was used to define recovery). Only words that were used at least 15 times were represented in the content analysis. The content analysis method was emergent (“open language” or “bottom up”), meaning that word frequencies were derived from participant responses and compared, rather than supplied in categories a priori.
Results
Pretraining measure assessments were undertaken by 591 staff members (81%). The results are summarized in
Table 1.
Initial Training Measures
Of the sample, 277 staff members returned at least one matched pre- and posttraining measure assessment. We observed statistically significant improvements in staff knowledge, attitudes, skills, and self-confidence. Wilcoxon signed rank test results for change in outcome scores for pre- and posttraining measures are shown in
Table 2.
Booster Training Measures
Booster training was provided to 176 staff members; 36 booster training measure assessments were matched with pre- and posttraining measures. The results were examined for overall differences by using the Friedman test, as shown in
Table 3. Post hoc analysis (with Wilcoxon signed rank tests) indicated that staff attitude, skills, and self-confidence showed an initial, statistically significant increase. Attitude and self-confidence remained unchanged at the booster session, although skills decreased. Knowledge, perceived importance of CRM implementation, and confidence in the organization did not change over time in this smaller group.
We analyzed the language used in staff definitions of recovery pre- and posttraining to assess any change in staff’s understanding of recovery and development of terminology and shared language during the initial training period. In total, 7,790 words from 613 definitions of recovery (pretraining=376, posttraining=237) were entered. The highest frequency of use for a given word was 178 uses, and the lowest was 15. Use of medicalized language was reduced by >50%; the frequency of the word “illness” decreased from 55 to 19 uses, and “symptoms” from 25 to 10 uses. Recovery language showed a corresponding threefold increase in frequency, as indicated by “wellbeing” increasing from 25 to 75 uses and “resilience” from 16 to 56 uses (see word cloud in the online supplement).
Discussion
The results of this study indicate that CRM staff development training in a large public mental health service improved knowledge, attitudes, skills, and confidence in implementing CRM practice among multidisciplinary staff members. Staff ratings did not increase for perceived importance of CRM implementation or confidence in the organization implementing CRM. Staff definitions of recovery revealed the development of shared language across the organization. Results of analyses of follow-up measure assessments at booster training indicated maintenance of initial improvements in staff attitudes and self-confidence in their ability to implement CRM. Mean rank scores measuring staff skills decreased below the initial training score. A high proportion of the workforce participated in CRM training and evaluation. Cofacilitation and coproduction were central to this successful training model. To our knowledge, these results represent the largest CRM data set collected and published by a single clinical organization (
17,
20). The findings of this study indicate that it is possible to deliver impactful CRM training to a whole workforce providing services for patients across the life span and across settings and disciplines.
Initial Training Changes
The initial improvements in knowledge, attitudes, skills, and self-confidence appeared to be a direct result of the initial 3-day CRM training. These findings are consistent with those of smaller studies that have shown improved knowledge (
20) and attitudes (
19) after implementation of CRM training. Improvement in staff skills has been demonstrated by other recovery training programs (
32,
33). The findings of the present study align with those of a recent large Danish study examining the impact of a recovery training program that also utilized coteaching (
34).
Change enhancement is central to CRM. To better understand staff’s motivation to change from a medical to collaborative recovery model, we investigated the perceived importance of and confidence in CRM implementation. Confidence was investigated as both staff’s confidence in their own and in their organization’s ability to implement CRM. This motivation for change required a self-rated confidence score of approximately 70% and an importance score as high as possible (
30). To evaluate organizational implementation, both confidence in staff’s own practice (pretraining mean=68%) and confidence in organizational change (pretraining mean=75%) were measured. Staff’s self-confidence increased with each training session. Although scores for confidence in the organization did not increase, they continued to reflect readiness for implementing CRM in clinical practice. Perceived importance of implementing CRM was maintained at >80%, indicating staff readiness for change.
This study is notable for a whole-service approach that included medical staff. Engagement of medical staff is known to be a key driver of successful recovery framework interventions (
12); failure to engage medical staff has limited the efficacy of implementation in other studies (
35). These findings reflect the importance of a whole-workforce strategy in changing an illness-focused culture.
Language Matters
Attention to the use of language is a focus of recovery-oriented practice implementation (
10). The posttraining content analysis revealed a reduced heterogeneity in words, indicating an increased shared vocabulary. The reduced number of discrete words indicated harmonization of clinicians’ use of language, and the words used indicated recognition of recovery as an individual process.
The words used posttraining were more aligned with the guiding principles and components of CRM. This change in language likely reflected changes in the understanding of recovery and in shared narrative and meaning. We noted the reduced use of medicalized language, such as “symptom” and “illness,” and increased use of “wellbeing” and “resilience,” which in CRM convey the meaning of recovery. The increased use of harmonized and positive language was congruent with the largest qualitative recovery training analysis to date, recently conducted in Denmark (
34). The findings of our study support the effectiveness of CRM training to develop staff use of language that is person centered, strengths based, and future focused (
17).
The Value of Booster Training
Although our results suggest that booster sessions supported sustained improvement in attitude and self-confidence, the strength of this finding was limited by the small number of matched measures and the absence of a control group. The lack of a change in perceived importance of CRM implementation may reflect factors noted in other studies, such as institutional constraints (
16), competing service implementation priorities (
36), and the negativity bias within a clinical service (
37). Coaching in team-based reflective practice (
23) may have further contributed to the positive changes in attitudes, practice, and culture identified at booster training. Although model fidelity over time has been demonstrated by others (
38), the results of the present study suggest that booster training may assist this process.
Strengths and Limitations
The most significant limitation of this study was loss of posttraining and booster training data. This loss occurred because of a low rate of training completion, unmatched data pairs, and staff attrition. The low training completion rate was related to wavering commitment to CRM and to competing priorities. The study was conducted by using an anonymous code system for matching, which led to unexpected difficulty matching participants at the 6-month booster training. The significant attrition rate has implications for both the certainty and generalizability of the findings.
We acknowledge the likely discrepancy between staff’s self-reported attitudes, actual beliefs, and enacted behaviors. It is unclear whether the changes in staff self-reports led to meaningful behavioral change among staff or experiential change for consumers and families; this is an important area of future inquiry. The study would have been strengthened by the use of a control group; however, the practical nature of this study meant that all staff were required to be trained.
To the best of our knowledge, this study provides the largest published qualitative and quantitative data set evaluating the implementation of CRM in a single organization across multiple service types and disciplines in a large public mental health service. This study achieved a higher completion rate on CRM training measures than did previous studies investigating the effectiveness of CRM training (
19,
20). Key study strengths were cofacilitation and the aspiration for coproduction in efforts toward coplanning, codesign, codelivery, and coevaluation of CRM.
Future Research
Other factors pertaining to CRM implementation warrant investigation, such as qualitative evaluation by participants and trainers and the impact of training on different disciplines, programs, and service settings. Further investigation of factors that affect CRM acceptability, uptake, and change in practice is needed. Future research should evaluate the effects of recovery-oriented practice on consumers by using the Self-Identified Stage of Recovery tool (
39) and Consumer Evaluation of CRM (
40). The studied impacts of staff recovery training on consumer and caregiver experience and outcome vary from weak (
41,
42) to strong (
35) and warrant further investigation. Training that affects organizational culture and staff attitudes may further support successful introduction and integration of a lived-experience workforce (
43). Future research should focus on the effectiveness of CRM in the lives of consumers, their families, caregivers, and other support people and on their experience of mental health services.
Conclusions
CRM was implemented to the whole workforce of a large public mental health program in metropolitan Melbourne. The implementation was cofacilitated and was intended to maximize opportunities for coproduction within organizational constraints. This study provides promising preliminary evidence that a CRM staff development program delivered in this way enhances staff knowledge, attitudes, skills, confidence, and language usage in delivering recovery-oriented practice and delivers sustained changes in attitudes and self-confidence. Future evaluation should explore the mechanisms that support the sustainability of CRM practice and outcomes and the qualitative experiences of consumers, their families, caregivers, and other support people.
Acknowledgments
The authors acknowledge all collaborative recovery model (CRM) trainers, other members of the CRM workforce development team, and Margie Nunn, B.A., Elizabeth Fraser, B.S.W., Michael Gardner, M.B.B.S., M.P., and Kevin Ong, M.B.B.S., for their contribution to the CRM training and implementation and the development of this article. The authors also thank Neami National, Australia, the licensed provider of CRM training, which provided train-the-trainer training, trainer mentoring, and implementation guidance. CRM was developed at the University of Wollongong, Australia.