A Review of Mental Health Interventions in SAMHSA's National Registry of Evidence-Based Programs and Practices
Abstract
Objective:
This study identified and examined characteristics of mental health interventions listed in a registry of evidence-based behavioral health programs and practices.
Methods:
Data were from the National Registry of Evidence-Based Programs and Practices (NREPP), which is sponsored by the Substance Abuse and Mental Health Services Administration. All mental health interventions posted as of May 2010 were reviewed.
Results:
A total of 91 interventions were assessed. Over half (57%) focused on promoting mental health rather than on treating mental illness. The implementation materials and trainings for most interventions (52%) were proprietary. Analysis of NREPP's rated dimensions revealed that quality-of-research scores were significantly lower for interventions with exclusively proprietary materials, whereas readiness-for-dissemination scoreswere significantly lower for interventions with exclusively publicly available materials.
Conclusions:
NREPP represents a practical tool for organizations seeking to initiate or expand their use of evidence-based services. Periodic studies of listed interventions may help to identify programmatic gaps in NREPP. (Psychiatric Services 62:303–305, 2011)
Despite recent recommendations promoting greater adoption of evidence-based interventions in mental health (1,2), relevant information useful to practitioners and organizations interested in identifying, comparing, selecting, and implementing such interventions has been limited. Yet the need for practical information on a range of evidence-based mental health interventions will likely intensify as states, communities, and other funders encourage or even require that agencies and providers routinely implement these services (3,4).
The National Registry of Evidence-Based Programs and Practices (NREPP), developed by the U.S. Substance Abuse and Mental Health Services Administration (SAMHSA), is emerging as an objective and credible source of practical information on a range of evidence-based interventions to prevent and treat mental and substance use disorders. As described on its Web site (www.nrepp.samhsa.gov), the purpose of NREPP is to “assist the public in identifying approaches to preventing and treating mental and/or substance use disorders that have been scientifically tested and that can be readily disseminated to the field.” To further this purpose, NREPP provides descriptive information on a range of interventions, as well as independent, expert ratings of both the quality of research to support an intervention's key outcomes and the intervention's readiness for dissemination to broader service settings and populations. Since launching NREPP in March 2007, SAMHSA has been committed to the steady and continuous growth of NREPP as a decision support vehicle to assist individuals and agencies in determining which evidence-based interventions best address their needs, resources, and organizational cultures. The NREPP Web site averages more than 13,000 unique visitors per month.
This review identified and assessed the range of mental health interventions currently listed on NREPP. We highlight key characteristics of these interventions and—informed by these findings—briefly discuss the future implications that an evolving NREPP may have on research, services delivery, and mental health policy.
Methods
We conducted a search of the NREPP database to identify interventions that were posted to the Web site between March 1, 2007, and May 17, 2010, and that were classified as mental health promotion (preventing or delaying the onset of emotional or behavioral problems), mental health treatment (services aimed at reducing or eliminating symptoms or impairments from mental illness), or both. (Interventions related to substance use disorders were excluded.) For all interventions that met these criteria, further descriptive information was gathered regarding the age focus of the intervention (child and adolescent, adult, or both); the setting of the intervention (clinic based, school based, or other, such as home, community, corrections, and workplace); whether the intervention evaluation had been replicated; and the status of the intervention as proprietary (fees are charged by developers for access to implementation materials or trainings), publicly available (no fees are associated with access to implementation materials or trainings), or a mixture of proprietary and publicly available materials and trainings.
Rating information was also gathered on all mental health interventions and included the average quality-of-research score for all cited intervention outcomes, and the average readiness-for-dissemination score. Both quality of research and readiness for dissemination are scored on a scale of 0 to 4, with 4 representing the highest possible score on any given criterion. More detailed information on the criteria for both the quality-of-research and readiness-for-dissemination scores is available on the NREPP Web site.
Results
At the time the study was conducted, 91 of the 159 interventions (57%) listed on NREPP met inclusion criteria as either a mental health promotion or a mental health treatment intervention (or both). The descriptive characteristics of these interventions are presented in Table 1. Mental health interventions listed on NREPP were more likely to focus on promotion than treatment and on children and adolescents than adults. The interventions also were more frequently delivered in nonclinic settings (for example, schools, community settings, and homes), had undergone one or more replications, and were proprietary in nature.
Proprietary status, in particular, was associated with several other variables. Quality-of-research scores for key intervention outcomes were lower for interventions with exclusively proprietary materials and trainings (2.82±.55) than for interventions for which the materials or trainings were either all publicly available (3.23±.53) or a mix of proprietary and publicly available (3.14±.50) (F=4.91, df=2 and 88, p<.010); a Scheffé post hoc test indicated that the scores for the proprietary and public programs were significantly different from one another.
In contrast, readiness-for-dissemination scores were lower for interventions with exclusively publicly available materials (2.42±1.00) than for interventions for which the materials or trainings were either exclusively proprietary (3.30±.69) or a mix of proprietary and publicly available (3.22±.76) (F=6.63, df=2 and 88, p<.002); a Scheffé post hoc test indicated that the scores for proprietary and public programs were significantly different from one another.
Compared with interventions focused on adults, those focused exclusively on youths were more likely to be proprietary (χ2=6.98, df=2, p<.030). Whereas 24 (68%) child-adolescent interventions were proprietary, only eight (33%) adult interventions and 15 (50%) combined adult and child-adolescent interventions were proprietary.
Age also appeared to be associated with the setting of the intervention. Interventions for adults were more likely to take place in a traditional mental health setting (inpatient, outpatient, or residential) than those for either children and adolescents or for both adults and children (χ2=18.07, df=2, p<.001). Eighteen (75%) adult interventions took place in a traditional mental health setting, whereas nine (22%) child-adolescent interventions and ten (33%) combined adult and child-adolescent interventions took place in such a setting.
Discussion
This study provides an initial snapshot of mental health interventions contained in an important federally sponsored registry of evidence-based programs and practices. Results indicate that among mental health interventions currently included in NREPP, a larger percentage focus on the promotion of mental health than on the treatment of mental illness. The interventions emphasize a range of ages. Most interventions—especially youth-focused programs—are delivered partly or fully outside of traditional mental health settings. An overwhelming majority of the mental health interventions currently listed on NREPP are partly, if not fully, proprietary in nature—that is, the developers charge interested parties for some or all of the materials or trainings needed to successfully implement the intervention.
The finding that most child and adolescent interventions are proprietary may be a result of the inclusion in NREPP of some interventions from a predecessor registry that identified “model” substance abuse prevention and mental health promotion programs. Interventions from this system that met the NREPP criteria were identified and rereviewed for inclusion in NREPP, and many of these are proprietary school-based programs. Also, the finding that most adult interventions are designed to be delivered in traditional mental health settings is consistent with historical mental health service delivery patterns (5).
Perhaps the most striking finding is the relationship between dimension scores and program status. In general, interventions for which the necessary materials or trainings are partly or exclusively proprietary have lower quality-of-research scores but higher readiness-for-dissemination scores than interventions for which the materials or trainings are publicly available. This finding may reflect researchers' different motivations and financial incentives related to developing and marketing their interventions. In other words, this finding may suggest that some researchers—or their designated agents—become “purveyors” or marketers of a particular intervention and thus may focus less on expanding the intervention's evidence base than on facilitating the intervention's dissemination (6). Such a finding is worthy of further exploration, as it may reveal important factors that might influence the manner in which evidence-based mental health interventions are developed and disseminated in the future (7).
Nevertheless, the NREPP submission and selection processes represent important limitations of the study reported here. Interventions listed in the NREPP database have been voluntarily nominated for inclusion, and the extent to which those included are representative of all mental health interventions—or even of interventions that have been evaluated—is uncertain. Moreover, to be selected for review, interventions must meet certain minimum requirements, which are identified in annual guidance published by SAMHSA.
Legitimate questions remain as to whether the NREPP submission or selection process contributes in any systematic way to greater inclusion or exclusion of certain types of mental health interventions. Attention to these issues will be important because resource limitations, among other factors, preclude NREPP from becoming an exhaustive repository of all tested mental health interventions.
The continually evolving nature of NREPP represents another important limitation. As the registry grows by the addition of programs, aggregate data on the characteristics of included interventions are likely to change. However, the dynamic nature of NREPP also underscores the importance of periodic assessments similar to or even more extensive than this study. Such efforts might enable SAMHSA to more readily identify programmatic gaps in the overall NREPP portfolio that could be addressed during the annual open submission process for soliciting new interventions. A recent example of this during the 2010 open submission period was a call for interventions with publicly available implementation materials and trainings.
Conclusions
Many challenges confront organizations interested in successfully implementing evidence-based programs. Committed leadership, motivated and competent staff, congruence of the program with the organization's mission and values, and adequate financial resources are but a few of the drivers necessary to deliver on the promise offered by evidence-based practices (8,9). Given the complexities associated with implementing evidence-based interventions, it is not surprising that many community-based mental health organizations have yet to widely embrace their use.
Nevertheless, NREPP represents a practical tool for behavioral health organizations seeking to initiate or expand their use of evidence-based services (10). The information and ratings contained in NREPP may help agencies and others determine whether an intervention can help them to achieve desired outcomes for the individuals and families they serve. Moreover, as NREPP continues to evolve, efforts such as this study may identify gaps in the mental health services evidence base toward which future research funds might be directed.
Acknowledgments and disclosures
The authors report no competing interests.
References
1.
Institute of Medicine: Improving the Quality of Care for Mental and Substance-Use Conditions. Washington, DC, National Academies Press, 2006
2.
Achieving the Promise: Transforming Mental Health Care in America. Pub no SMA-03-3832. Rockville, Md, Department of Health and Human Services, President's New Freedom Commission on Mental Health, 2003
3.
Tanenbaum SJ: Expanding the terms of the debate: evidence-based practice and public policy; in Evidence-Based Psychotherapy: Where Practice and Research Meet. Edited by, Goodhart CD, Kazdin AE, Sternberg RJ Washington, DC, American Psychological Association, 2006
4.
Cooper JL, Aratani Y: The status of states' policies to support evidence-based practices in children's mental health. Psychiatric Services 60:1672–1675, 2009
5.
Duffy FF, Wilk J, West JC, et al. Mental Health Practitioners and Trainees in Mental Health, United States, 2004. Rockville, Md, Substance Abuse and Mental Health Services Administration, Center for Mental Health Services, 2008
6.
Fixen DL, Naoom SF, Blasé KF, et al. Implementation Research: A Synthesis of the Literature. Tampa, Fla, University of South Florida, Louis de la Parte Florida Mental Health Institute, 2005
7.
Leff S, Conley J, Hennessy K: Marketing behavioral health: implications of evidence assessments for behavioral health care in the United States. International Journal of Mental Health 35:6–20, 2006
8.
Rosenberg L: The reality of implementing evidence-based practices. Journal of Behavioral Health Services and Research 37:1–3, 2010
9.
Phillips SD, Allred CA: Organizational management: What service providers are doing while researchers are disseminating interventions. Journal of Behavioral Health Services and Research 33:156–175, 2006
10.
Hennessy KD, Finkbiner R, Hill G: The National Registry of Evidence-based Programs and Practices: a decision-support tool to advance the use of evidence-based services. International Journal of Mental Health 35:21–34, 2006
Figures and Tables
Information & Authors
Information
Published In
Cover: Spring Thaw, by Ernest Lawson, circa 1910. Oil on canvas, 25¼ × 30 inches. Terra Foundation for American Art, Daniel J. Terra Collection. Photo credit: Terra Foundation for American Art, Chicago/Art Resource, New York.
History
Published online: 1 March 2011
Published in print: March 2011
Authors
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBGet Access
Login options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).