Homelessness among young adults is a serious problem worldwide (
1,
2). Even though the estimation of homeless young adults in the Netherlands varies widely, the minimum estimate is 9,000 (
2). Although this population experiences a wide range of problems, including abuse and trauma (
3), addiction (
4), and general medical and mental health problems (
5,
6), scientific evidence for effective interventions targeted at improving quality of life for this group is scarce (
7). Moreover, there is a gap between homeless young adults’ expressed needs and the support provided by professionals (
8,
9). In the Netherlands, a strengths-based intervention for these youths, called Houvast (the Dutch word for “grip”), has been developed in collaboration with professionals and homeless young adults and manualized for standardized training and supervision of mental health professionals. The intervention is based on the strengths model as developed by Rapp and Goscha (
10) and has been tailored to the homeless young adult population. The main aim of Houvast is to improve the quality of life of these youths by focusing on their strengths and stimulating their capacity for self-reliance.
Over the past decade, there has been considerable methodological progress in the measurement of fidelity (
10), as demonstrated by the development of fidelity scales for different models, such as assertive community treatment and the strengths model (
10,
13). Several studies have revealed that effective interventions compared with others tend to be characterized by higher fidelity scores that produce better client outcomes (
11,
12,
14–
19). Houvast is a strengths-based intervention for which a fidelity scale is available (
10). This article describes the fidelity assessment of the Houvast intervention in five shelters for homeless young adults.
Methods
Participants
As part of a study of the effectiveness of Houvast, fidelity assessment of this strengths-based intervention was conducted among five Dutch shelters for homeless young adults. These facilities provide ambulant care (one facility), residential care (two facilities), or both (two facilities) to homeless young adults ages 18 to 26. More details on the effectiveness study can be found elsewhere (
20).
Informed consent was obtained from homeless young adults as well as professionals. The study fulfilled the criteria for approval by an accredited Medical Review Ethics Committee within the region Arnhem-Nijmegen. Upon consultation, the Ethics Committee stated that because of the behavioral character of the intervention, the study (registration number 2011/260) was exempt from formal review.
Introduction and Implementation of Houvast
Houvast is based on the principles of the strengths model and uses the tools specified for the model. A theoretical framework and trajectory were elaborated, and training material, including new tools, was developed for working with homeless young adults. [Further information about Houvast and the implementation activities we used is presented in an online supplement to this article. This supplement also contains baseline demographic characteristics of the 117 participants and additional information about a progress evaluation held six months after the audit.]
In fall 2011, we introduced Houvast at all five shelters that were randomly allocated to the experimental condition of the study. Upon consenting to participate, two to three managers of each facility attended a meeting in which additional information on the study was provided, as well as guidelines to optimize the implementation of Houvast. Team leaders, who are responsible for team work and the daily organization of tasks at the shelters (such as creating work schedules), attended a two-day training session provided by certified trainers. Team leaders were taught how to support professionals in adhering to Houvast and to maintain quality in implementation, for example through group supervision meetings. In addition, all professionals who are responsible for the daily support of homeless young adults received a four-day training course in the same period (October 2011–January 2012). Six days of training (April 2012–May 2012) were provided to supervisors, who were part of the team and were responsible for providing feedback to professionals to attain fidelity to the Houvast model. Besides training in the basic aspects and competencies of supervision, supervisors were taught to review the Houvast tools, such as the strengths assessment and the personal recovery plan. Finally, in September 2012, all professionals attended a one-day follow-up training session.
Strengths Model Fidelity Scale
Fidelity was measured with the strengths model fidelity scale, developed and validated by Rapp and Goscha (
21).
Table 1 shows the Dutch strengths model fidelity scale, the data sources, and the instruments we used for assessment (
10). The scale consists of ten indicators distributed across three subscales—structure, supervision, and clinical practice—and includes professionals’ responsibilities, caseload ratio, group supervision, supervisor duties and caseload, strengths assessment and its integration, use of personal recovery plans, community contact, use of naturally occurring resources, and use of hope-inducing behavior. Quality requirements are listed for each indicator.
Each indicator is rated on a 5-point scale, as is commonly used in other fidelity research (
22). For example, the item “Strengths assessment is used to help clients develop treatment plan goals” is rated as 1, up to 60% of the files; 2, 61%−70% of the files; 3, 71%−80% of the files; 4, 81%−90% of the files; or 5, 91%−100% of the files. A total fidelity score was obtained by averaging the scores on the ten indicators. A score of ≥4 indicates sufficient fidelity and <4 indicates insufficient fidelity (
13,
21).
Fidelity Assessment
To acquire a fidelity score, we translated and adjusted several fidelity assessment tools and methods, such as interview outlines, an observation scheme for group supervision, and questionnaires. Two trained assessors who had no relation to the teams conducted a one-day audit at each of the five facilities. During a four-day training course, assessors were taught how to conduct the audit, obtain measurements, and write a standardized report. During this training, the assessors’ scores on indicators were discussed until consensus was reached, and during the audit they obtained complete interrater agreement on the rating of group supervision.
Each audit included the following activities: observation of a group supervision meeting, a file analysis of a maximum of 12 files of six randomly chosen professionals whose clients received care for at least one month (N=46), five focus group interviews with three to four homeless young adults (N=19) who received care in the shelter for at least one month, and a separate interview with the team leader (N=5) and the supervisor (N=4) to gather additional information about their respective roles and the implementation of Houvast. Two weeks before the audit, the professionals, supervisors, and team leaders were asked to fill out a questionnaire on the use of Houvast (such as their use of personal recovery plans and naturally occurring resources). The homeless young adults who participated in the focus groups received care at the shelter for five to 47 weeks on average. Audits were conducted between June and September 2012.
Each shelter received a report on the audit, which included scores on the indicators, the total fidelity score, a summary of the results, and specific recommendations for improving model fidelity. Within six weeks after the audit, we held a conference call with the team leader, the supervisor, and the manager to evaluate the audit and to discuss the fidelity results and recommendations.
Booster Sessions and Progress Evaluation
On the basis of results of the fidelity scales, booster sessions were organized for the whole team (April–June 2013). During a booster session at each of the five shelters, a certified trainer discussed the recommendations of each fidelity report, and attendees practiced key elements of Houvast, for example by conducting role plays. Before the booster sessions were held, we evaluated the progress in fidelity since the audit. Team leaders and supervisors were asked whether the indicators of the fidelity scale (except hope-inducing behavior) had improved, deteriorated, or stayed the same compared with the audit in 2012. [Results are presented in the online supplement.]
Discussion
This study is the first to report on the assessment of fidelity to Houvast, a strengths-based intervention for homeless young adults. The scores on three of the ten indicators of fidelity were sufficient six months after the introduction of Houvast at the shelters and after professionals and team leaders received training in the intervention: caseload ratio, group supervision, and strengths assessment. The total score for the five participating shelters for homeless young adults showed insufficient fidelity to Houvast, but the fidelity assessment provided the participating facilities with concrete guidelines for improvement.
This study showed that much still needs to be done to attain fidelity to the Houvast model. How can we explain the apparent gap between the application of Houvast in practice and model fidelity? This study marked the first time that the Dutch version of the strengths model fidelity scale was used, and its validity for use in the Netherlands may need further analysis. Although in this study the data were too limited to investigate the quality of the scale, the results constitute a first indication for face validity because the participating professionals, team leaders, and supervisors acknowledged the scale’s indicators and perceived the fidelity scores as credible. Furthermore, the auditors considered the fidelity scale to be a useful tool for assessing fidelity to the Houvast model, they reached consensus on the indicators’ scores, and they did not encounter any problems in applying the scale despite differences between the American and Dutch context.
From questionnaires filled out by professionals and from interviews with the supervisors and team leaders, we found that professionals were using Houvast tools inconsistently at the primary process level and seemed to have some difficulty with integrating the process with other training and as part of their daily routine. Some professionals, for example, regarded the personal recovery plan as less useful or as just another form to fill out instead of as something meaningful for their homeless young clients. Also, during the focus groups with homeless young adults and on the basis of information extracted from their completed questionnaires, we found that many professionals did not model hope-inducing behavior as Houvast requires. Perhaps they needed more time to adopt and express hope-inducing competencies, given that doing so requires a shift in attitude and not merely a behavioral change to be authentic. Moreover, supervisors mentioned that their workload was too high; thus not every professional received field mentoring. Also, because supervisors received their training after the professionals and team leaders in the first four months, supervisors’ skills may not have been optimal for ensuring implementation. Furthermore, professionals reported using institutionalized resources instead of naturally occurring resources. Institutionalized resources are services or facilities that are not available for every citizen in the community, such as mental health care and substance use treatment. In contrast, naturally occurring resources are services or facilities available for every citizen in the community, such as neighbors, friends, and clubs.
The paradigm shift professionals needed to make became clear from information retrieved from the interviews with supervisors and team leaders and the focus groups with homeless young adults: Houvast requires professionals to focus on the strengths and talents of homeless young adults rather than on their problems, even though this was not part of professionals’ formal training or daily routine. This shift required professionals also to let go of their control over service delivery and the working relationship and to become a coparticipant rather than the expert in the recovery process of homeless young adults (
23,
24). The questions professionals asked their supervisors, such as “What [can I] do with personal goals of homeless young adults that seem unrealistic?” indicate that the essentials of Houvast had not yet been fully adopted.
From questionnaires, interviews, and conference calls with supervisors and team leaders, we learned that there were also factors at the organizational level that inhibited the adoption of Houvast and that may have negatively influenced the scoring on all subscales. First, responses often indicated that time constraints made obtaining a sufficient fidelity score on indicators such as supervision and community contact nearly unachievable. Consequently, most supervisors provided supervision when professionals experienced difficulties in their work. Also, professionals were not always able to work in the community because the work schedule for the shelter facilities required them to work on site.
Second, in most organizations only the professionals working with homeless young adults at the same facilities received training in Houvast, whereas other professionals in the same organization working at other facilities did not receive Houvast training and worked according to a more problem-oriented approach. These differences, in turn, led to conflicting demands and expectations of the professionals using the Houvast approach. At one shelter, professionals had to work with Houvast tools as well as with the tools of the organization.
Third, some team leaders or supervisors mentioned reorganizations and financial reductions, which may have caused teams to be unstable and experience high turnover of employees. Although training was offered to new staff, few Houvast-trained professionals remained part of some teams.
Fourth, the Houvast tools were incorporated in the electronic file systems of only a few organizations. At facilities where the tools were not incorporated, professionals needed additional registration time for updating client records, and this time was not always available. More registration time was often necessary because the health insurers require problem-oriented registration to fund care, whereas the Houvast tools do not generate this type of information because of the strengths base of the approach.
For these reasons, the adoption of Houvast by all those involved in the implementation process was difficult to achieve in the six-month period between the introduction of Houvast and the fidelity assessment. What can be learned about implementing a strengths model? This model is more than a collection of tools; it is a philosophical approach that requires organizationwide adoption (
25). Providing supervision of implementation of new practices is essential; to lead the effort, supervisors should be trained before professionals. Team leaders should also receive training (
25).
In the six-month period, the implementation of Houvast focused mainly on the internal infrastructure of the facility (working according to the tools of Houvast) and captured neither the entire organization nor relevant external relations. As confirmed by supervisors and team leaders, training should be conducted not only for all professionals working with homeless young adults in the same organization but also for colleagues in the organization who are working with other groups and for those performing other tasks, such as working at the reception desk (
25). An infrastructure that supports the implementation and maintenance of Houvast over time is essential to improve model fidelity (
25,
26). In addition, the availability of financial resources and the willingness of organizations to adopt the strengths-based approach and to take the necessary measures to make that happen (such as investing in supervision) are factors that could boost all fidelity indicators (
27–
29). Previous studies have corroborated the value of supportive leadership at multiple levels (
30–
32), and agency and program directors who can facilitate implementation (
32) are essential for model fidelity. Furthermore, greater perceived benefit (
33) and a high level of congruence between organizational values and characteristics of an intervention would facilitate implementation (
32,
34).