How Can the Use of Evidence in Mental Health Policy Be Increased? A Systematic Review
Abstract
Objective:
Methods:
Results:
Conclusions:
Methods
Data Sources and Search Strategy
Classification
Exclusion Criteria
Establishment of Domains
Intervention Studies
Results
Methods of the Intervention Studies
Study | Design | Sample selection and size | Method | Data collection methods | Timing of data collection | Outcome measures |
---|---|---|---|---|---|---|
Chamberlain et al., 2008–2012 (30,31,38,48) | Randomized controlled trial | 56 counties in California and Ohio | County clusters were randomly assigned into 1 of 2 conditions and 3 timeframes (cohorts 1, 2, and 3). In the individual condition, counties implemented the evidence-based practice (MTFC) individually in the absence of community development teams. For the community development teams, groups of 4–8 counties were assembled into teams and helped to engage in peer-peer networking, as well as receiving technical assistance and support from consultants. | Self-report surveys, document review, video data (to monitor fidelity), activity log (for example, contact logs between the system leaders and study staff, number of foster homes available, number of foster parents recruited), independent assessment (for example, trainer impressions of foster parents or staff) | Data collected across the 10 intervention components | Stages of Implementation Completion measure, which uses multiple data sources to assess completion of multiple stages of implementation; time taken for counties to complete multiple stages of MTFC implementation; time taken to place children in an MTFC-based foster care |
Glisson et al., 2005–2013 (17,29,43,44,49) | Randomized controlled trial | 14 counties from the Appalachian region of Tennessee. To be eligible, a county could not fall within a metropolitan statistical area, had to contain primarily communities with <2,500 residents and no communities with >15,000, had to have a lower per capita income than the state average ($28,641), and had to have a greater proportion of children living in poverty than the state average (17.6%). | 2 × 2 factorial design. The ARC intervention was randomized at the county level; 6 of 14 counties were allocated to receive ARC. Then young people referred to juvenile court for delinquency in all 14 participating counties were randomly assigned to receive a new evidence-based practice (multisystemic therapy) or usual care. ARC involves a series of activities guided by an ARC specialist to create support within the community for services for the target population (in this case, delinquent youths), help service organizations facilitate improvements in service delivery, and develop social networks among key individuals (opinion leaders, stakeholders, and service providers). About 30% of ARC specialist time is spent with community stakeholders. ARC addresses challenges to implementation with 3 strategies: providing organizational tools needed to identify and overcome service barriers (for example, teamwork and goal setting), introducing principles of effective service systems, and enhancing service provider behaviors and attitudes that are conducive to service improvement efforts. | Therapist logs, caregiver reports | Therapist log completed weekly and collected weekly; TAM-R completed monthly; MST audio coding (3 sessions per family from early, mid, and late treatment); CBCL completed at baseline, 6, 12, and 18 months; number of youths in out-of-home placements identified via monthly phone calls with caregivers and in-depth interviews at baseline, 6, 12 and 18 months | Multisystemic therapist logs, TAM-R by phone, multisystemic therapy audio coding adherence system, caregiver report, CBCL, youths in out-of-home placements |
Chamberlain et al., 2012 (31) | Rolling cohort | 18 of 20 teams of local authorities and agencies in England that won funding to establish MTFC | National implementation team assisted successful grantees in implementing MTFC. This involved providing guidance and assistance at all levels, including setting up multiagency teams, helping to make the systems-level changes needed to implement the program, providing training for staff and foster parents, and providing ongoing supervision and regular audit and feedback sessions. | Details not provided | Annually for each of the 4 years of the grant | Details not provided |
Feinberg et al., 2002 (37) | Cross-sectional | 203 community leader participants (for example, from government, human services, law enforcement, and schools) from 21 communties in Pennsylvania | Community leaders received training in prevention science during the 1-year planning period for Communities That Care (CTC) program implementation. Three training sessions were held: key leader orientation, 2 days; risk and resource analysis, 3 days; promising approaches training, 3 days. | Structured interview with the CTC program director collecting qualitative and quantitative data for each of the 21 local coalitions; then interviews with 9 key leaders from each of the 21 communities | Program directors, 2 years after the training intervention; key leaders, 4 years after; no pretraining measures | Attendance at training (self-reported and from register); comprehensive, structured interview for key leaders and program directors covering the following domains: readiness, attitudes and knowledge, CTC external relations, CTC functioning, CTC efficacy |
Driedger et al., 2010 (32) | Case study | 9 data analyst–manager dyads, who were part of an earlier mapping project, selected from Ontario Early Years Centres in Ontario, Canada | 2 training sessions for data analysts in the use of EYEMAP mapping software and other mapping software if required and ongoing support and advice; 4 training sessions for analyst-manager dyads on how to interpret spatial data and how to apply these data to policy and program development work | Field notes, e-mail exchanges between the research team and participants, self-report, focus groups, qualitative interviews | Pre-, mid- and postintervention | Mapping skills assessment, qualitative data (focus groups, interviews, e-mails, field notes) |
Luck et al., 2009 (33) | Case study | Audience segmentation used to target segments on a national (U.S.) level, including decision makers, clinicians, frontline workers, and consumers; sample size not specified | Social marketing approaches adapted to key segments. For national-level decision makers: research summaries, cost-benefit estimates, brochures, meetings, and teleconferences; for service-level decision makers: implementation plans, training, training materials, clinical evidence summaries, small group meetings, conference invitations, and meetings to discuss customization; for frontline workers and clinicians: evidence summaries, case studies and testimonials, training and procedures, presentations by practitioners; for potential consumers: presentations by Veterans Service Organization representatives and information from other veterans who had engaged with the program | Not specified | Details not provided | Details not provided |
McGrath et al., 2009 (34) | Case study | Key stakeholders in child mental health in Nova Scotia, Canada; sample size and method of selection not specified | Integrated knowledge translation strategy. Researchers engaged in numerous dissemination strategies in 3 main phases of the project: design, research, and study completion. For the design stage, feedback on program development, design of materials, cost-effectiveness, and service delivery was sought from key stakeholders. For the research phase, key stakeholders were contacted regularly and provided with feedback regarding the trial (costs, progress, and risk management). Community dissemination strategies were undertaken (such as mall visits, study launches, community exhibits, and TV campaigns) to promote Family Help. For the study completion phase, researchers presented study findings at relevant conferences. | A number of qualitative observations offered, but no details provided on how these data were collected | Details not provided | Details not provided |
Stark et al., 2013 (35) | Case study | Community mental health service teams from the U.K. National Health Service (NHS) Highland were targeted because of a knowledge exchange partnership between the University of Stirling and the NHS Highland; 10 of 13 teams participated; 13 family caregivers and 7 people with dementia were interviewed. Eligibility criteria were not described. | Researchers from the University of Stirling and executives from the NHS Highland partnered to facilitate knowledge translation and production of locally relevant dementia research. Steering group included NHS executives, researchers, service staff, and charity representatives. Monthly meetings covered decision making, meeting national dementia targets, and identifying local priorities. A systematic review of rural dementia service literature was conducted. Observational, case study, and consultation data were collected to identify current practices of service staff and knowledge of policy makers. Data plus the literature review findings were used to develop and implement strategies to increase knowledge or skills related to dementia management. | Data from surveys, observation, consultations, and interviews used to measure preliminary outcomes, practices, and knowledge; data then guided further initiatives; unclear how main outcomes were assessed | During and immediately after the 2-year project | A survey of community mental health services to identify current practice; observation of practice in community, clinic, and ward settings; consultation with service users and caregivers |
Ward et al., 2012 (36) | Case study | 3 management or service delivery teams in a U.K. mental health organization | A knowledge broker was engaged to help each team formulate a plan to address a particular service delivery or evaluation challenge. The broker’s work addressed 3 components of knowledge exchange: information management, linkage and exchange, and capacity building. | Qualitative data collected from knowledge broker field notes and narrative interviews conducted by an independent researcher with members of the teams | Knowledge broker field notes collected throughout the trial; narrative interviews conducted at the end of the trial | Qualitative data; no formal outcome measures used |
Strategies to Increase the Use of Evidence
Study | Strategies tested | Target group and policy level | Level administered | Frequency of intervention | Duration of intervention | Decision maker or broker involved | Funding source | Results |
---|---|---|---|---|---|---|---|---|
Chamberlain et al., 2008–2012 (30,31,38,48) | Stimulating better communication and relationships among stakeholders; increasing the extent to which the organization and staff value research evidence; increasing the extent to which staff have the knowledge and skills to use evidence; increasing the extent to which the organization has the tools and systems needed to support research engagement and use; increasing access to research evidence; increasing the interaction between decision makers and researchers | Decision makers, clinicians, frontline workers, and consumers at the regional level | Individual and group | Ongoing | Not stated | Decision makers | Government policy agencies | Trial still in progress |
Chamberlain et al., 2012 (31) | Stimulating better communication and relationships among stakeholders; increasing the extent to which the organization and staff value research evidence; increasing the extent to which staff have the knowledge and skills to use evidence; increasing the extent to which the organization has the tools and systems needed to support research engagement and use; increasing the interaction between policy makers and researchers | Decision makers, frontline staff, and foster parents at the national level | Individual and group | Details not provided | 4 years | Decision makers | Government policy agency | No data on formal study outcomes were provided. However, several sites received grants and established multidimensional treatment foster care, and positive system changes reportedly occurred. |
Driedger et al., 2010 (32) | Increasing the extent to which the organization and staff value research evidence; increasing the extent to which staff have the knowledge and skills to use evidence; increasing the generation of new research or analysis by policy makers | Decision makers and analysts at the regional level | Both individual and dyad | Ongoing | Approximately 2 years | Decision makers | Publicly funded research agency | Most analysts exhibited increased mapping skills. Qualitative data indicated some increase in the use of maps to support decision making. |
Feinberg et al., 2002 (37) | Increasing the extent to which the organization and staff value research evidence; increasing the extent to which staff have the knowledge and skills to use evidence; increasing the extent to which the organization has the tools and systems needed to support research engagement and use; increasing access to research evidence; increasing the generation of new research or analyses by decision makers; increasing the interaction between decision makers and researchers | Decision makers at the state level | Group | Ongoing | 12 months | Decision makers | Government agency | Key leader training was associated with leaders’ more positive perception of the internal and external functioning of the coalition. Some evidence was found that the intervention may improve individual attitudes and knowledge. |
Glisson et al., 2005–2013, (17,29,43,44,49) | Stimulating community support; stimulating better communication and relationships among stakeholders; increasing the extent to which the organization and staff value research evidence; increasing the extent to which staff have the knowledge and skills to use evidence; increasing the extent to which the organization has the tools and systems needed to support research engagement and use | Decision makers, clinicians, frontline workers, and consumers at the regional level | Individual and group | Ongoing | Approximately 40 months | Decision makers | Publicly funded research agency | At 6-month follow-up, youths total problem behavior score was significantly lower among those in the group receiving multisystemic therapy and Availability, Responsiveness and Continuity (ARC) intervention, compared with youths in the other conditions. No difference was found between groups at 18-month follow-up in terms of problem behavior, but youths in the group receiving multisystemic therapy plus ARC were significantly less likely to have entered out-of-home placements. |
Luck et al., 2009 (33) | Stimulating community support; stimulating better communication and relationships among stakeholders; increasing the extent to which the organization and staff value research evidence; increasing access to research evidence; increasing the interaction between decision makers and researchers | Decision makers, clinicians, frontline workers, researchers, and consumers at the national level | Group | Details not provided | Details not provided | Decision makers | Government policy agency | The depression collaborative care model promoted was adopted by the Veterans Health Administration as part of the new priority health initiative and associated policies. |
McGrath et al., 2009 (34) | Stimulating community support; stimulating better communication and relationships among stakeholders; increasing the extent to which the organization and staff value research evidence; increasing the extent to which staff have the knowledge and skills to use evidence; increasing access to research evidence; increasing the interaction between decision makers and researchers | Decision makers, clinicians, frontline workers, researchers, and the general public at the state level | Group and individual | Ongoing | Approximately 7 years | Decision makers | Government policy agency, publicly funded research agency | Outcome data were not reported. Anecdotally, it was reported that Family Help obtained wide acceptance from users and had an influx of referrals. Funding and services for Family Help expanded to additional districts, and there was increased interest in the program from other provinces. |
Stark et al., 2013 (35) | Increasing the extent to which the organization and staff value research evidence; increasing the extent to which staff have the knowledge and skills to use evidence; increasing access to research evidence; increasing the interaction between decision makers and researchers | Decision makers, clinicians, frontline workers, and researchers at the regional level | Group level | Details not provided | 2 years | Decision makers | Government policy agency | Outcome data were not reported. Anecdotal findings indicated positive policy changes, including increased prioritization for dementia care through development of diagnostic clinics, commissioning a dementia training strategy, implementation of national care standards and diagnostic standards across all operational areas, and a recommendation that all community mental health teams implement an agreed-upon, integrated care pathway. Funding was also provided to dementia charities to support patients and families during early diagnosis. |
Ward et al., 2012 (36) | Increasing the extent to which the organization and staff value research; increasing the extent to which staff have the skills and knowledge to engage with and use research; increasing access to research evidence | Decision makers and clinicians at the regional level | Group | Ongoing | 10–15 months | Decision makers and brokers | Publicly funded research agency | Use of all 5 components of knowledge exchange was found to increase over the study period. Components were not discrete and often co-occurred. |
Policy influences.
Capacity.
Research engagement actions.
Impact of Strategies to Increase the Use of Evidence
Discussion
Conclusions
References
Information & Authors
Information
Published In
History
Authors
Funding Information
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).