Provider Attitudes and Experience With Measurement-Based Mental Health Care in the VA Implementation Project

We developed a survey of provider knowledge and attitudes regarding measurement-based care. The questionnaire was developed with a focus on Collect, Act, Share—the three concepts that are central to the VA’s implementation of measurement-based care. Drafts of the instrument were tested with several providers and then refined and distributed to another group of providers (N=25), who also provided feedback. The final instrument consists of 32 items, including 29 items scored on a Likert scale and three items scored as categorical.

The survey was distributed to mental health providers at 58 sites in 2017 as a quality improvement component of the measurement-based care initiative; thus institutional review board approval was not required. Each site identified a local champion. The survey was sent to the champion and then distributed locally; thus the number of providers who were invited to participate is unknown. Surveys were anonymously completed using REDCap, an online data collection system. Because the goal of the survey was to capture baseline use and attitudes about measurement-based care, the survey was distributed prior to any educational efforts to increase use of measurement-based care.

Rates of measurement-based care usage at each facility prior to the initiative were available by query from the electronic health record in administrative data records. To be consistent with the initiative, we defined use of measurement-based care as repeated use (two or more occasions) of the same patient-reported outcome measure within the first 6 months of a new episode of care. The four measures counted were those emphasized by the initiative, including the Patient Health Questionnaire–9 (PHQ-9) (12), the Generalized Anxiety Disorder Screener (GAD-7) (13), the Brief Addiction Monitor (BAM) (14), and the PTSD Checklist (PCL-5) (15). A new episode of care was defined as having two or more visits with an ICD-10 code for depression within a 6-month period after having had no visits with such codes in the preceding 2 years, with at least one of the visits occurring in a specialty mental health clinic. The index date was the first time the code was used.

Moreover, to be counted as measurement-based care, use of the instrument had to be recorded in the electronic health record in a way that was accessible to other providers (sharable). Sharable data were defined as data recorded by a method that could be graphed, assimilated, or easily searched. In the VA, that meant that the scores were recorded in the measures data file (Mental Health Assistant [MHA] file) and not solely recorded as text in a progress note. Because the PHQ-9 was the most frequently used measure, we report on rates of use of the PHQ-9 in the treatment of depression as a proxy for use of measurement-based care at the participating sites.

Analyses were limited to frontline providers who spent at least 10 hours each week in clinical care. All scores were recoded so that higher scores indicated positive outcomes. Analyses were conducted by using SAS and consisted of chi-square and analysis-of-variance statistics.

At 11 of the 58 sites, no providers participated in the survey. In total, 296 surveys were returned. We excluded 22 results submitted from licensed counselors or other clinicians, given the small number and heterogeneity of providers in that group. An additional 44 providers were excluded because they had limited clinical time. This resulted in an analytic group of 230 providers, with 26 identified as psychiatrists, 23 as nurses, 135 as psychologists, and 46 as social workers.

More than half of the providers (N=134, 58%) reported collecting at least one measure for at least half of their patients. Nearly half (N=101, 44%) reported collecting measures repeatedly during an episode of care. Psychiatrists were least likely to collect measures, followed, in order of least likelihood, by nurses, social workers, and psychologists (χ²=54.9, df=240, p<0.001). For example, 16% of psychiatrists, 44% of nurses, 57% of social workers, and 71% of psychologists reported collecting at least one measure for at least 50% of patients. Similarly, psychiatrists were least likely to collect measures repeatedly (χ²=37.7, df=229, p<0.004), with 8% reporting having collected measures repeatedly for at least 50% of patients, compared with 30% of nurses, 44% of social workers, and 54% of psychologists.

Several survey items were related to the utility of measurement-based care in the care of patients (Table 1). In general, providers were positively oriented toward the utility of patient-reported outcomes. There were few differences in attitudes about the utility of measures by provider type.

Providers frequently shared the results of outcome measures with patients, with 72% (N=166) indicating that they usually or always explained the results to patients and 65% (N=150) indicating that they used the results as part of shared decision making. However, only 37% (N=86) indicated that they used the scores when discussing a patient with another provider. The percentage of providers who shared patient-reported outcomes varied by provider type (explained results to patients, χ²=28.3, df=229, p=0.005; shared results with patients in order to accomplish shared decision making, χ²=28.5, df=229, p=0.005; and shared results with other providers, χ²=23.8, df=229, p=0.02). Sharing result also varied by provider type. Psychiatrists reported sharing least compared with all provider types. [A table reporting the extent to which providers shared patient-reported outcomes is available as an online supplement to this report.]

In additional responses, providers indicated a low degree of worry that results from patient-reported outcomes can be used by supervisors to evaluate provider performance (mean±SD score=7.5±2.7, with higher scores indicating less worry); there were no differences by provider type on this item. Further, all providers believed that research shows that patient-reported outcomes are effective (7.4±2.1, with higher scores indicating greater belief in effectiveness). However, there were disparities in the degree to which providers felt informed about using the results in their practice (6.5±0.4, psychiatrists; 6.8±0.4, nurses; 7.8±0.3, social workers; and 8.9±0.2, psychologists; with higher scores indicating having felt more informed (p<0.001).

At the 47 participating sites, 4.6% (±4.2) of patients received two or more PHQ-9 assessments in the first 6 months after starting an episode of depression care. Use of measurement-based care for depression at the participating sites was no different than use in the other 93 medical centers in the VA system.

Results from this survey demonstrated relatively high acceptance among VA providers for the use of measurement-based care in the treatment of mental illness, with several accompanying themes. In this sample of selected VA providers, however, there was significant disparity across various disciplines, with psychiatrists reporting having collected, shared, and used the measures less often than members of other disciplines, especially psychologists, who often reported higher scores on attitudes toward and participation in measurement-based care compared with other professionals. Psychologists may be more predisposed than psychiatrists to using measurement-based care, given that many training programs in psychology emphasize the importance of measurement-based care and measurement of outcomes is part of evidence-based psychotherapy, such as cognitive processing therapy or cognitive-behavioral therapy. The findings are consistent with results from surveys of other providers, which also show low engagement in measurement-based care among psychiatrists (16, 17). Furthermore, although sharing results with patients was common, providers were less likely to consider the importance of sharing results with colleagues. Finally, despite the positive responses of providers, patient electronic health records at the participating sites indicated relatively low overall use of measurement-based care in the treatment of major depression.

Measurement-based care is becoming an expectation of many stakeholders. Accreditation programs such as the Joint Commission, insurers, quality management agencies, and patients are increasing asking that care be driven by patient-reported outcomes. The results of this provider survey suggest that progress has been made with some providers, but this cultural shift in managing patients’ care needs continual attention. Many providers who have embraced measurement-based care have found that using measures makes their session more efficient and focused and provides the patients with a voice that they did not have prior to using measures. But for providers who have not been trained that way, changing their practice requires a lot of faith and effort. These results should offer some strategies to address implementation, such as focusing on demonstrating clinical utility to psychiatrists, educating about the value of sharing results with other providers, and studying ways to more efficiently collect patient-reported outcomes. Another facet of adoption of measurement-based care is the infrastructure needed to support routine measurement. Only recently have there been efforts to develop information technology platforms that make it easier to collect, use, and share measures easily in routine care.

Limitations of this work included not knowing the completion rate among staff who were given the survey and the focus on providers already engaged in delivering measurement-based care. It is highly likely that selection bias favored providers who use or endorse measurement-based care. This may also account for the relatively low participation by psychiatrists, nurses, and social workers in this survey. Also, the survey was given only to a small group of highly selected providers and thus may not generalize beyond these settings. There is also the possibility of social desirability in responses, given that providers knew how they “should” answer even if it didn’t reflect their practice. The contrast between favorable attitudes and knowledge about measurement-based care and its low use at the facility where providers work could be a reflection of provider bias or the difficulty of sharing patient-reported outcome, given the current state of informatics. If there are technological barriers for easily recording measures in the medical record, providers may limit their use to paper and simply record total scores in the record progress note text rather than enter measure data into the measures file (e.g., MHA). So, although many providers may use and share outcomes with patients, they may not go the extra step to share outcomes in the electronic health record in a way that other providers can easily access.

The results of this survey are encouraging, but they also point to a needed cultural shift in the way mental health care is delivered. Providers expressed favorable attitudes for using measures in clinical practice, but these attitudes varied significantly by discipline. While sharing measurement data with patients is commonly reported, measures need to be shared with other providers in order to drive care and create a narrative about a patient over time in the electronic health record. When sharing of scores with patients and providers is achieved, we will have realized a major step forward in reducing the stigma of receiving treatment for mental illness.