State Adoption of Incentives to Promote Evidence-Based Practices in Behavioral Health Systems
Abstract
Objective:
Despite the critical role behavioral health care payers can play in creating an incentive to use evidence-based practices (EBPs), little research has examined which incentives are used in public mental health systems, the largest providers of mental health care in the United States.
Methods:
The authors surveyed state mental health directors from 44 states about whether they used any of seven strategies to increase the use of EBPs. Participants also ranked attributes of each incentive on the basis of key characteristics of diffusion of innovation theory (perceived advantage, simplicity, compatibility, observability, and gradually implementable) and perceived effectiveness.
Results:
Almost three-quarters of state directors endorsed using at least one financial incentive; most paid for training and technical assistance. Few used other incentives. Strategies perceived as simple and compatible were more readily adopted. Enhanced rates and paying for better outcomes were perceived as the most effective but were the least deployed, suggesting that simplicity and organizational compatibility may be the most decisive factors when choosing incentives.
Conclusions:
Payers are not using the incentives they perceive as most effective, and they are mostly using only one strategy for reasons of simplicity and compatibility. Future work should focus on barriers to measurement that likely hinder the adoption and implementation of paying for better outcomes and enhanced reimbursement rates, with the ultimate goal of measuring the effectiveness of incentives on EBP implementation efforts.
Many public behavioral health systems are actively trying to increase the availability of evidence-based practices (EBPs) for the people they serve. A rapidly expanding list of state and county systems encourage agencies to use EBPs by offering them incentives or instituting mandates. Free training and technical assistance are often provided as well (1,2). Despite this momentum and the growing consensus that EBPs are critical to providing quality care, they are still used infrequently (3,4). One explanation for this enduring research–practice gap is that the policies used to encourage the implementation of EBPs are themselves not effective (5).
Financing and establishing regulatory policy are important drivers of EBP use in almost every published conceptual framework of factors affecting implementation (6). Raghavan and colleagues (5) have suggested that payers have at their disposal the following policy mechanisms to increase the use of EBPs within agencies: implementing enhanced reimbursement strategies for the use of EBPs, training providers and consumers in EBPs, publicly recognizing high-performing agencies, reducing regulatory burden, giving agencies that use EBPs advantages in competitive bidding processes (fast-tracking), paying for better outcomes, and directing patient traffic toward agencies using EBPs. In 2003, the President’s New Freedom Commission on Mental Health urged the public sector to provide financial incentives to increase the use of EBPs (7). Few of the other incentives described by Raghavan and colleagues have been widely promoted, and there has been almost no measurement of whether and how incentives are used in practice (8–10).
State behavioral health systems are in a unique and powerful position to promote the use of EBPs. Although ultimately we are interested in how financial incentives may promote the use of EBPs, we must first establish the extent to which different incentive strategies are used. In this study, we gathered information about states’ use of each incentive and state officials’ perceptions of the characteristics of these incentives. We characterized these incentives as innovations that states choose to adopt or not adopt and therefore used diffusion of innovation (DOI) theory (11) as a framework. The DOI framework has been used across varied disciplines (for example, agriculture, medicine, and information technology) to understand the adoption of new innovations or practices. It posits five key characteristics of an innovation that influence the decision to adopt: perceived advantage, compatibility with existing procedures, simplicity, gradually implementable, and observable. Perceived advantage is the benefit that the innovation appears to provide over current procedures. Compatibility reflects the congruence between the innovation and existing procedures. Simplicity describes how readily alterations in extant procedures could be accommodated.
DOI theory posits that an innovation is more likely to be accepted if it can be gradually implemented in small steps on a trial basis (trialability) and if the results are observable, or visible to others. Moreover, these key attributes are critical factors that encourage the adoption of any innovation—in this case, innovations to promote EBP. In his synthesis of diffusion studies, Rogers (11) suggested that perceived advantage, simplicity, and compatibility are key factors that explain adoption decisions, whereas observability and trialability are not as consistently important. We therefore hypothesized that incentives that hold relative advantage and are simple and compatible would be more likely to be implemented.
Methods
Participants and Procedures
We partnered with the largest organization of state mental health directors, the National Association of State Mental Health Program Directors, to distribute a brief survey. We sent an e-mail inviting directors (or their designated proxies) to complete a survey on incentives and EBPs. Several reminders were sent. Data were collected from representatives of 44 states (response rate=88%), representing 90% of the U.S. population (12). Data were collected between October 2016 and April 2017. All procedures were approved by the University of Pennsylvania Institutional Review Board.
Measures
Our survey instrument consisted of 21 items. Efforts were made to minimize the number of items to enhance the response rate. Respondents were queried on their use of seven different innovations described by Raghavan and colleagues (5) and the probability that they would implement each incentive in the future to promote the use of EBPs. Respondents were asked to endorse whether each incentive met the criteria for each of the five DOI characteristics (perceived advantage, compatibility with existing procedures, simplicity, gradually implementable, observable). Finally, respondents were asked to endorse (on a five-point scale) how effective each incentive would be in promoting the adoption of EBPs if the barriers to these incentives were removed. The survey was developed and field tested in consultation with an advisory board of stakeholders who had administrative and research experience in public mental health systems.
Statistical Analysis
For each incentive, we calculated the percentages for states’ use, endorsement of each of the five DOI attributes, and perception of perceived effectiveness (N=44; see Table 1). We then examined the association between these seven summary statistics with Pearson product–moment correlation coefficients to assess the relationships between endorsement of each DOI key attribute and use of incentives.
| DOI characteristicb | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Implementation | Better than what we are doing now to promote EBPs | Compatible with existing procedures and infrastructure | Simple to implement | Gradually implemented in small steps or on a trial basis | Effects immediately observable | Effectivenessc | ||||||||
| Incentive type | N | % | N | % | N | % | N | % | N | % | N | % | M | SD |
| Paid for training and technical assistance | 27 | 61 | 6 | 14 | 29 | 66 | 42 | 95 | 8 | 18 | 7 | 27 | 3.79 | .94 |
| Publicly recognized high-performing EBP providers | 10 | 23 | 12 | 27 | 22 | 50 | 34 | 77 | 12 | 27 | 5 | 10 | 3.29 | .77 |
| Directed patients to providers who use EBPs | 8 | 18 | 7 | 16 | 13 | 30 | 30 | 68 | 9 | 20 | 4 | 8 | 3.64 | .96 |
| Enhanced reimbursement rates | 5 | 11 | 18 | 41 | 14 | 32 | 31 | 70 | 22 | 50 | 4 | 5 | 4.40 | .80 |
| Reduced regulatory burden for providers who provide EBP | 4 | 9 | 8 | 18 | 7 | 16 | 16 | 36 | 13 | 30 | 7 | 4 | 3.55 | .94 |
| Paid agencies for better clinical outcomes | 4 | 9 | 15 | 34 | 10 | 23 | 23 | 52 | 23 | 52 | 9 | 4 | 4.24 | .98 |
| Fast-tracked providers using EBPs in the competitive bidding process | 1 | 2 | 2 | 5 | 7 | 16 | 18 | 41 | 9 | 20 | 1 | 1 | 3.24 | 1.23 |
a
Results are based on a survey of state mental health directors.
b
Respondents were asked to endorse whether each incentive met the criteria for five characteristics in the DOI theory framework.
c
Respondents were asked to rate how effective the incentive would be if it were easily implemented, from 1, not effective, to 5, very effective.
Results
Overall, 71% of states endorsed using at least one incentive to promote the use of EBPs. Table 1 shows that 61% paid for training and technical assistance. Fewer than one-quarter reported using each of the other strategies: publicly recognizing high-performing EBP providers (23%), directing patients to providers who use EBPs (18%), enhanced reimbursement rates (11%), paying agencies for better clinical outcomes (9%), and fast-tracking providers using EBPs in the competitive bidding process (2%). The two incentives rated as most likely to be effective if implemented were enhanced reimbursement rates and paying for better outcomes.
Figure 1 displays the correlations between the use of incentives and the DOI constructs. Perception of the innovation’s compatibility with existing procedures and its use were positively correlated (r=.92, p=.003). There was also a positive correlation between simplicity and use (r=.85, p<.017).
FIGURE 1. Correlations between states’ use of incentives for adoption of evidence-based practices (EBPs) and the incentives’ characteristics and perceived effectivenessa
aResults are based on a survey of 44 state mental health directors. The characteristics are from the diffusion of innovation theory framework. A significant correlation was found between use of incentives and compatibility (<.01) and simplicity (p<.05).
Discussion
The majority of expenditures in the behavioral health marketplace are publicly funded (13). State governments, the largest payers in behavioral health, therefore have much at stake in designing and implementing effective incentives to improve care. We found that the most commonly used incentive to increase the use of EBPs providing training and technical assistance. Few states do anything else, despite the growing evidence and consensus that, as the primary vehicle of EBP implementation efforts in the public sector, training is necessary but not sufficient (14,15). Training alone does not lead to change in practice (16,17)
On average, state directors rated training as both compatible and simple to use. However, it was rated as much less effective and much lower on all other DOI attributes than enhanced reimbursement rates and paying for better outcomes, which were among the least used strategies. This finding suggests that, of the DOI constructs, simplicity and compatibility may be particularly important in driving the adoption of financial incentives to promote EBPs in public behavioral health systems.
We have two hypotheses about why enhanced reimbursement rates and paying for better outcomes are not used, despite their being rated as effective and advantageous. First, of the seven incentives, these two require by far the most measurement. The adoption of enhanced reimbursement rates and paying for better outcomes to promote EBPs, although simple in concept, are extraordinarily complex in execution. They require sophisticated data collection and analysis, and payers must have the financial capacity to pay for incentives (18). Although there are many validated measures of symptoms and functioning, there are no definitive and few widely accepted behavioral health outcome measures, such as the presence or absence of disease or mortality. Some have argued that incentives, metrics, and populations have not been adequately designed or identified to ensure (and measure) quality improvement through incentives (19–21). Helping states implement enhanced rates and paying for outcomes will first require more research and support to help these systems implement comprehensive and easy-to-use approaches to measuring outcomes.
Second, enhanced reimbursement rates and paying for better outcomes have the potential to create inequity in the public behavioral health care system, with some providers benefiting more than others, which can in turn affect providers’ bottom line, fracture relationships between payers and providers, or even reduce access to care if providers leave the network. Any implementation in a behavioral health care system requires a delicate dance of collaboration, coordination, and negotiation among all stakeholders, particularly given rapidly contracting state behavioral health budgets (22,23). Collaborative stakeholder approaches to identifying fair and transparent incentives would be particularly beneficial to this end (24). Future research that includes qualitative study might allow for a more detailed assessment of and insight into these challenges while identifying specific barriers that may be common across settings.
It is important to acknowledge some limitations of this research. First, state mental health directors and their designated proxies may not be fully informed about all state behavioral health initiatives, localized efforts, or pilot projects. The turnover in state leadership amplifies this potential missed information (personal communication, Hepburn B, 2017). We also recognize that in some states, the county may be a more meaningful unit of analysis in terms of the role of payment, policy, and management, and county-level representation was not included in this study. In addition, state policies and financing change frequently, particularly since passage of the Affordable Care Act. The findings presented in this article are a snapshot of an evolving process as it appeared in 2016.
Conclusions
Despite these limitations, this study contributes new information about the financial incentives used by the biggest players in the behavioral health marketplace to promote EBPs. To our knowledge, this is the first national survey of state-level payers regarding the financial incentives they use to increase the use of EBPs. Payers are not using the incentives they perceive to be most effective, and they are largely only using one strategy for reasons of simplicity and compatibility. Future work should focus on barriers to measurement that likely hinder the adoption and implementation of enhanced reimbursement rates and paying for better outcomes, with the ultimate goal of measuring the effectiveness of incentives in EBP implementation efforts.
References
1.
Hoagwood KE, Olin SS, Horwitz S, et al: Scaling up evidence-based practices for children and families in New York State: toward evidence-based policies on implementation for state mental health systems. Journal of Clinical Child and Adolescent Psychology 43:145–157, 2014
2.
Beidas RS, Stewart RE, Adams DR, et al: A multi-level examination of stakeholder perspectives of implementation of evidence-based practices in a large urban publicly-funded mental health system. Administration and Policy in Mental Health and Mental Health Services Research 43:893–908, 2016
3.
McHugh RK, Barlow DH: The dissemination and implementation of evidence-based psychological treatments: a review of current efforts. American Psychologist 65:73–84, 2010
4.
Bruns EJ, Kerns SEU, Pullmann MD, et al: Research, data, and evidence-based treatment use in state behavioral health systems, 2001–2012. Psychiatric Services 67:496–503, 2016
5.
Raghavan R, Bright CL, Shadoin AL: Toward a policy ecology of implementation of evidence-based practices in public mental health settings. Implementation Science 3:26, 2008
6.
Damschroder LJ, Aron DC, Keith RE, et al: Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Science 4:50, 2009
7.
Achieving the Promise: Transforming Mental Health Care in America. Pub no SMA-03-3832. Rockville, MD: New Freedom Commission on Mental Health, 2003
8.
Ducharme LJ, Abraham AJ: State policy influence on the early diffusion of buprenorphine in community treatment programs. Substance Abuse Treatment, Prevention, and Policy 3:17, 2008
9.
Rieckmann TR, Kovas AE, Cassidy EF, et al: Employing policy and purchasing levers to increase the use of evidence-based practices in community-based substance abuse treatment settings: reports from single state authorities. Evaluation and Program Planning 34:366–374, 2011
10.
Rieckmann TR, Kovas AE, Fussell HE, et al: Implementation of evidence-based practices for treatment of alcohol and drug disorders: the role of the state authority. Journal of Behavioral Health Services and Research 36:407–419, 2009
11.
Rogers E: Diffusion of Innovations, 5th ed. New York, Simon & Schuster, 2003
12.
State Population Totals Tables: 2010–2016. Washington, DC, US Bureau of the Census, 2017. https://www.census.gov/data/datasets/2017/demo/popest/state-total.html
13.
Garfield RL: Mental Health Financing in the United States: A Primer. Washington, DC, Kaiser Commission on Medicaid and the Uninsured, 2011 https://kaiserfamilyfoundation.files.wordpress.com/2013/01/8182.pdf
14.
Rapp CA, Bond GR, Becker DR, et al: The role of state mental health authorities in promoting improved client outcomes through evidence-based practice. Community Mental Health Journal 41:347–363, 2005
15.
Herschell AD, Kolko DJ, Baumann BL, et al: The role of therapist training in the implementation of psychosocial treatments: a review and critique with recommendations. Clinical Psychology Review 30:448–466, 2010
16.
Beidas RS, Kendall PC: Training therapists in evidence-based practice: a critical review of studies from a systems-contextual perspective. Clinical Psychology: Science and Practice 17:1–30, 2010
17.
Rakovshik SG, McManus F, Vazquez-Montes M, et al: Is supervision necessary? Examining the effects of Internet-based CBT training with and without supervision. Journal of Consulting and Clinical Psychology 84:191–199, 2016
18.
Stewart RE, Lareef I, Hadley TR, et al: Can we pay for performance in behavioral health care? Psychiatric Services 68:109–111, 2017
19.
Jha AK: Time to get serious about pay for performance. JAMA 309:347–348, 2013
20.
McKethan A, Jha AK: Designing smarter pay-for-performance programs. JAMA 312:2617–2618, 2014
21.
Garnick DW, Horgan CM, Acevedo A, et al: Influencing quality of outpatient SUD care: implementation of alerts and incentives in Washington State. Journal of Substance Abuse Treatment 82:93–101, 2017
22.
Honberg R, Kimball A, Diehl S, et al: State Mental Health Cuts: The Continuing Crisis. Arlington, VA, National Alliance on Mental Illness, 2011
23.
Stewart RE, Adams DR, Mandell DS, et al: The perfect storm: collision of the business of mental health and the implementation of evidence-based practices. Psychiatric Services 67:159–161, 2016
24.
Meredith LS, Branstrom RB, Azocar F, et al: A collaborative approach to identifying effective incentives for mental health clinicians to improve depression care in a large managed behavioral healthcare organization. Administration and Policy in Mental Health and Mental Health Services Research 38:193–202, 2011
Information & Authors
Information
Published In
Cover: Sea Grasses and Blue Sea, by Milton Avery, 1958. Oil on canvas. Gift of friends of the artist, Museum of Modern Art, New York. Digital image © The Museum of Modern Art/Licensed by SCALA/Art Resource, New York. © The Milton Avery Trust/Artists Rights Society, New York.
History
Received: 20 November 2017
Accepted: 19 December 2017
Published online: 1 March 2018
Published in print: June 01, 2018
Keywords
Authors
Competing Interests
Dr. Marcus reports receiving consulting fees from Allergan, Alkermes, Johnson & Johnson, and Sunovion. The other authors report no financial relationships with commercial interests.
Funding Information
National Institute of Mental Health10.13039/100000025: NIH 1F32MH103960
This research was funded by National Institutes of Health grant 1F32MH103960. The authors thank the directors who participated in the study and Ted Lutterman for his assistance with participant recruitment.Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).