Use of Quality Measures for Medicaid Behavioral Health Services by State Agencies: Implications for Health Care Reform

We conducted online searches of 50 state Medicaid and mental health and substance abuse agency Web sites in September 2013. Using the following terms, we searched the state Web sites in regard to behavioral health quality indicators: behavioral health quality; behavioral health performance; and mental health quality, mental health performance reports, publications, data, and special projects. A search was also conducted for state Medicaid managed care contracts, quality strategies, quality improvement plans, quality and performance indicators data, annual outcomes reports, performance measures specification manuals, legislative reports, and Medicaid waiver requests for proposals. We used the Joint Commission’s definition for quality indicator: a quantitative measure that can be used to monitor and evaluate the quality of governance, management, and clinical and support functions that affect behavioral health outcomes (15). These measures were typically listed as such on the agency Web sites.

Through federal Medicaid managed care regulations (42 C.F.R. §438.200), CMS requires all states contracting with a managed care organization to have a written State Quality Strategy for assessing and improving the quality of the managed care services they offer (16). Therefore, we searched for these CMS-required quality strategy reports. Finally, we searched for any surveys or similar tools administered by the states to assess consumer satisfaction with quality of care.

We then created a database to organize the information collected. The database contained the following information on each quality indicator: state name, indicator name, originator (HEDIS, state, and so on), domain (as established by the National Inventory of Mental Health Quality Measures [NIMHQM]), category (structure, process, or outcome), source document, Web site from which the information was retrieved, and whether the state has a Medicaid managed care behavioral health program.

We queried a number of states that did not have publicly available quality indicators, and two reasons these states provided for not publishing behavioral health quality indicators on the Internet were that their quality measurement initiatives were in the development stages and that there was no state mandate to publish quality measurement initiatives.

From the 50 states reviewed, 29 states provided information on their Web site: Arizona, California, Colorado, Connecticut, Florida, Illinois, Iowa, Kansas, Kentucky, Louisiana, Massachusetts, Michigan, Nebraska, Nevada, New Hampshire, New Jersey, New Mexico, New York, North Carolina, Oklahoma, Oregon, Rhode Island, South Carolina, Tennessee, Texas, Utah, Vermont, Virginia, and Washington. Except for New Hampshire, each of the 29 states reviewed had some form of Medicaid-managed behavioral health care. Some states used an integrated plan that offered behavioral and general medical care through the same health plan; other states used a carve-out plan, in which behavioral health services were managed by a separate health plan (Table 1).

Table 2 provides information on the quality measure reporting that the 29 states use. The Donabedian quality-of-care framework guided us in identifying these measures as relating to structure, process, or outcome (17). In this framework, structure measures concern the attributes of the setting, human resources, financing, and organizational structure; process measures describe what occurs in giving and receiving care; and outcome measures refer to the effects of health care on the health status of patients and populations (18). Table 2 also provides information on consumer experience-of-care surveys that states are using with individuals who receive behavioral health services and information on the originator of the measure.

Many state behavioral health performance measure efforts are under way. State variation in the assessment of behavioral health care services has implications for determining the quality of current care and the impact of health care reforms.

Many states have continued to rely on the NQF-endorsed HEDIS measures to provide stakeholders with an assessment of the quality of these services. However, as the landscape of behavioral health care changes, we found that many states are incorporating, adapting, or developing additional measures to fill in gaps. Some alterations broaden the existing HEDIS measures. Examples include the expansion of existing HEDIS follow-up measures to assess the quality of follow-up care after hospitalization for substance abuse or after emergency department visits. Some states have attempted to fill perceived gaps in the NQF-endorsed measures, such as the addition of measures that address recovery progression, integrated care, and patient safety. States are also using administrative and nonadministrative data to create quality measures that cover a wide variety of domains. An example of a measure using administrative data is Iowa’s measure of the percentage of expenditures used to support consumer-run services. Examples of nonadministrative data measures include Kansas’ measure of the percentage of state-qualified providers of services to children with serious emotional disturbance, Louisiana’s measure of the number of persons served by evidence-based practices and by promising practices that have been implemented to fidelity.

Despite states efforts to develop additional quality indicators that promote evidence-based care and increased access to care, gaps in monitoring the quality of care continue to be found. For example, the use of outcome measures among states was limited. Although almost half of the states (48%) included at least one measure that addressed issues such as self-reported improvement in symptom severity or stable living environment, only 20% of all cataloged measures could be classified as outcome measures. A proportion of the outcome measures (21%) were hospital readmission rates, which could be considered to be a proxy measure of outcome. We consider outcome measures to be an area for expansion in quality measurement, although states may hesitate to include measures that do not account for illness severity. Although the use of behavioral health risk-adjustment models has accelerated in recent years, states may lack resources to collect the data needed to conduct appropriate risk adjustment (21). None of the states in our sample indicated the use of risk adjustment for any measure.

Greater efforts are also needed to develop additional standardized substance abuse treatment measures, given that Medicaid expansion is expected to significantly increase coverage for individuals with substance use disorders (22). Although some states have responded by adapting the few existing HEDIS measures to capture follow-up from substance abuse treatment, additional measures are needed for evaluating screening, integrated care, and use of evidence-based treatment. New York has attempted to fill the gap in substance use disorder treatment measures by developing a comprehensive array of measures for substance abuse treatment, follow-up care, and psychopharmacological treatment.

States also need to standardize measures of crisis services utilization. Many states reportedly struggle with finding psychiatric beds for individuals needing inpatient psychiatric care and with emergency department overuse because of the reduction in psychiatric hospital beds, lack of community-based services, and lack of insurance for behavioral health treatment needs (23). The ACA may ultimately reduce overuse by providing needed insurance coverage; however, standardized emergency department utilization and emergency department wait-time measures are needed in the interim as more states resort to managed care for their behavioral health services. The standardization and adoption of crisis services utilization measures—such as the emergency department utilization measures used by Louisiana or the treatment follow-up measures used by Illinois and Colorado after emergency department visits—could provide states with a means of comparison and assist in improving quality of care.

The ubiquity of experience-of-care surveys is noteworthy. Our findings reveal that states used several different consumer surveys, including CAHPS, MHSIP, and state-developed or state-approved surveys. Only one state (Oklahoma) used the ECHO, which was developed specifically for managed behavioral health care. There are likely two main reasons for states’ high utilization of experience-of-care surveys. Some surveys are required for federal funding. The MHSIP is currently part of the data requirements for block grants from the SAMHSA Center for Mental Health Services. States initially started reporting MHSIP data in 2002, and as of December 2012, all 50 states were reporting MHSIP results (24). Some experience-of-care surveys appear to be related to managed care implementation. Although experience-of-care surveys are not a federal requirement, many states use the survey results as a means to show compliance with Medicaid managed care regulations.

A standardized survey that would allow for national comparisons, particularly among states with behavioral health managed care, would be beneficial. In the absence of widespread adoption of the ECHO, one possibility includes developing standardized modules to add to the MHSIP survey that would be specific to managed care and substance abuse treatment. Standardized administration and sampling strategies would also help states compare the quality of their services with the quality in other states.

Finally, it should be noted that these findings are fairly similar to earlier scans of behavioral health quality measures, which revealed that most state Medicaid programs use HEDIS measures, consumer surveys, and some outcome measures (25,26). This relatively fixed state of behavioral health quality measurement seems to indicate that states may face challenges in meeting the increasingly demanding needs of a complex and evolving health care system, but there are some indications that measurement of behavioral health care quality is improving. For example, states such as Colorado, Kansas, and New York are developing a rich array of measures to meet their needs for performance measurement. Also, the number of NQF-endorsed measures continues to increase. It is expected that state adoption of these measures will follow. Also, the expansion of Medicaid managed behavioral health care is providing the impetus for states and behavioral health providers to improve their data collection systems, thereby improving their ability to collect performance measurement data.

We acknowledge limitations in our study. We limited our review to a convenience sample defined by state information available on the Internet. The information provided on the Internet for the 29 states reviewed may not allow for a comprehensive assessment. Also, our findings may be skewed toward more information for states with Medicaid managed care, because these states are required to publish and submit a quality strategy to CMS.

Under the ACA insurance expansions, behavioral health services should be more readily accessible to those seeking care. A more robust quality improvement infrastructure is needed as behavioral health service utilization increases in the United States to improve the overall quality of care, improve population health, and reduce the cost of high-quality health care. Standardized surveys and measures, new measures, standardized administration procedures, and comprehensive sampling strategies should be developed to allow for state-by-state comparisons and for the establishment of national benchmarks. Finally, there is a need for additional nationally endorsed measures in the areas of substance abuse, treatment outcome, and crisis services. States are developing their own measures in these areas to fill the current gap, which may make standardization of measures more challenging in the future. The National Quality Strategy should be considered as a foundation for future development of national benchmarks.