Research, Data, and Evidence-Based Treatment Use in State Behavioral Health Systems, 2001–2012

Although there is evidence that many states have invested in infrastructure supports to encourage and support EBT implementation, the impact on availability and reach of EBTs has been relatively small. In 2012, between 65% and 80% of states reported using the EBTs for adults examined in this study. However, the median numbers of clients served by states were only about 1,000 for each EBT, and penetration rates ranged from only 2% to 3%. Uptake of child EBTs was lower: only 25% to 50% of states reported use of the interventions studied, and median numbers of clients served were only 250 to 400, reaching only 1%−3% of youths with serious emotional disturbance. It is not possible from these data to explain the reasons for the discrepancy in use of adult- versus child-focused EBTs; however, the discrepancy could result from the long-standing trend in state behavioral health systems to invest more heavily in services for adults (27).

The results also pointed to large gaps between need and available EBTs for both adults and children. Some of this observed gap may be the result of study limitations. Because the NRI survey focuses on a small number of multimodal interventions for individuals with complex needs, data for a range of well-known interventions for individuals with less complex or intensive behavioral health needs (for example, cognitive-behavioral therapies and behavioral parenting interventions) were not available. Thus the actual number of EBTs being used across states is likely higher. In addition, there have been significant changes since 2001 in states’ authority; for example, the increasing penetration of managed care has led to less direct state control over services (18,28). Thus states have less authority to oversee EBT initiatives or may be unaware of EBTs that are being used through managed care.

Nonetheless, the scant uptake of EBTs found in this study is consistent with other studies of specific EBT implementation, which demonstrate significant challenges in getting research-based services installed in real-world systems (29–32). Results highlight the fact that the population entrusted to public mental health care is not deriving benefits from receipt of evidence-based therapeutic services. Although each of the interventions examined has been proposed as a strategy to meet the behavioral health needs of individuals with the most complex presentations, and all are tailored to clients served by specific public systems (for example, foster care for therapeutic foster care and juvenile justice–involved youths for multisystemic therapy), only a very small portion of individuals actually receive one of the EBTs.

Perhaps more worrisome, adoption by states, numbers served, and penetration rates were found to be flat or declining. Even if these interventions are merely considered indicators or proxies of EBT uptake in states, trends examined here suggest that after a burst of initial adoption and expansion in the early 2000s, EBT uptake has leveled off, if not diminished. This study also found similar “inverted U”-shaped trends in state EBT adoption drivers, including consensus building among stakeholders, state-led fidelity monitoring, and specific budget requests for EBTs.

Given that research-based strategies require investment in resources, both types of downward trends are not surprising given the magnitude of budget shortages that hit most states from 2007 to 2012. In fiscal year 2013, 13 SMHAs still had expenditures below 2007 levels (33), and these recent cuts occurred on top of historical and long-standing budget shortfalls that have consistently affected SMHAs (27).

Regardless, the trend in reduced state support for EBTs and their implementation is troubling, given that resources beyond standard fee-for-service payments are typically necessary for providers to implement EBTs well (34–36). Moreover, the goal of reducing overall public expenditures “downstream” (for example, corrections services, emergency room care, and hospitalization) (37) is theoretically linked to maintaining investment in EBTs that reduce the need for these expensive services. Thus the trend to limit EBT investment may ultimately result in increased state spending. In future analyses, we plan to use NRI data along with other data to examine the relationship between fluctuations in state fiscal investments and use of research-based strategies.

Some EBT implementation support strategies increased across the study period. Building EBTs into provider contracts showed the single greatest increase among the variables examined, from 43% of states in 2001–2004 to 70% in 2009–2012. This phenomenon was also found in a study by Finnerty and colleagues (38), which found that in 2007–2009 the majority of states opted to promote substance use disorder–related EBTs by encouraging providers to deliver these practices with contract funds.

States also reported increased use over the study period of financial incentives for provider EBT use. This may be an encouraging trend, because it has been highlighted as a proactive fiscal policy that encourages EBT adoption (38–40). However, the number of states reporting such incentives in 2012 was only 35%, barely more than the 31% that reported using the strategy in 2004.

Despite substantial improvements in IT since the turn of the millennium, the percentage of states implementing a statewide outcomes-monitoring system, integrating client data sets across agencies, producing directories of research or evaluation projects, modifying IT systems to support EBTs, and conducting research and evaluation did not change from 2001 to 2012. In 2012, only half of states reported having integrated data systems or a client outcomes-monitoring system. This is very troubling given broad consensus that true accountability is achieved not merely through investment in EBTs (41) but also through investment in data systems, data integration, and outcomes monitoring that promotes feedback, understanding of system functioning, and continuous improvement (2,42,43).

Unfortunately, however, to preserve services, SMHAs reported making their cuts in “administrative” expenses, such as data and research use, IT systems, outcomes monitoring, evaluation, and fidelity monitoring. Among evaluation and data use variables, only “funding an external research center or institute” showed an increase over time, from 13% to 31%. Although this could be interpreted as a positive trend, it is mirrored by a decrease in SMHAs’ internal operation of research centers, from 17% to 8%.

The study had some limitations. SMHAs are not the only systems that provide EBTs in a state. Vocational rehabilitation agencies may provide supported employment, and child welfare or juvenile justice agencies may support therapeutic foster care or multisystemic therapy. When responding to NRI-administered surveys, SMHA respondents may not have been fully informed about all state behavioral health initiatives or about localized efforts or pilot projects. In addition, estimates of the penetration rates for EBTs were based on general estimates provided by SMHA respondents of the overall number of youths with serious emotional disturbance or adults with serious mental illness, which may not accurately reflect actual rates of individuals eligible for these EBTs.

Another limitation is that because of the NRI’s funding mandate to focus on services funded by federal block grants, EBT data focus only on services for adults with serious mental illness or children with serious emotional disturbance, not on populations with other conditions or less intensive needs. The extent to which these findings are applicable to EBT implementation for other conditions, less intensive needs, or early intervention and prevention is unknown. In addition, although most of the EBTs studied are well defined and manualized, therapeutic foster care is a broader service type, which may have affected the accuracy of the findings and interpretation of utilization trends. Similarly, other important drivers and indicators of research and EBT use were not surveyed and thus could not be examined, including workforce support and training, the role of changing state leadership, differential costs associated with implementing EBTs, and shifts over time to managed care. These are all important areas for future research. Finally, the study aimed to examine national trends, and thus we present aggregate results across all states. Although beyond the scope of this study, future analyses should examine patterns of individual state trends and predictors of these variations.

State investment in EBT implementation infrastructure and use of data are critical and are only likely to grow in importance as health care reforms call for more effective services and greater quality and accountability. Results of this study suggest, however, that SMHA investment in EBTs and implementation infrastructure is flat or declining in many areas. Consistent, reliable, and valid measurement of these constructs will assist states in ensuring that their systems of services are capable of meeting the needs of the populations for which they are responsible. Such data may also serve as a basis for renewed investment and a metric for reform efforts.

The authors thank the hundreds of representatives of state mental health authorities for completing surveys over the years and thus making this study possible. They also thank the William T. Grant Foundation, whose initiative to study the use of research evidence in public systems inspired these analyses.