Skip to main content
Full access
Brief Reports
Published Online: 3 February 2021

A Multimethod Study of Mental Health Care Patients’ Attitudes Toward Clinician-Level Performance Information

Abstract

Objective:

Research demonstrates variability in mental health clinicians’ overall and domain-specific outcomes for their patients. Despite calls to increase patient access to provider performance information, little is known about patients’ attitudes toward and valuing of this information. This study explored patient attitudes, preferences, and values regarding the use of clinician outcome track records in provider selection and treatment decision making.

Methods:

Community mental health patients (N=403) completed a multicomponent survey, and a subset of patients (N=15) completed a follow-up semistructured interview. Interview data were analyzed with consensual qualitative methods.

Results:

Overall, participants endorsed valuing access to clinician outcome track record information and endorsed the belief that using such information would enhance the referral process and promote better mental health outcomes.

Conclusions:

Harnessing measurement-based information on clinicians’ effectiveness to make more personalized treatment decisions could promote better treatment engagement, retention, and outcomes.

HIGHLIGHTS

Patients’ perspectives on having access to clinician performance information are lacking from the research literature.
Survey and interview results indicated that patients would value access to clinician outcome track records to inform care decisions.
Patients believe that referrals informed by clinician track records would improve outcomes.
Research shows statistically significant and clinically meaningful differences among mental health care (MHC) clinicians in their patients’ outcomes (1). As assessed by standardized, patient-reported outcome (PRO) measures, some clinicians are consistently more than twice as effective as others, even after controlling for case-mix variables that influence outcomes (2). Clinician-level effectiveness also appears to be stable; clinicians who achieve better than average outcomes generally continue to achieve better than average outcomes with subsequent patient cohorts (3).
It is important for MHC stakeholders to consider how to make outcome information actionable to improve care quality. Implications of outcome variability already affect MHC and reimbursement decisions. The Centers for Medicare and Medicaid Services have instituted data-informed, value-based payment programs, recently extending requirements to psychologists. In addition to requiring collection and report of quality data, the payment program has established reimbursement levels on the basis of a clinic’s or individual provider’s performance relative to predetermined benchmarks (4).
The Institute of Medicine (IOM) (5) recommends that patients be granted access to clinician performance data to guide treatment decisions. However, patient perspectives on clinician performance track records remain conspicuously absent in MHC discussions. For example, do patients view this information as trustworthy and useful? Care decisions and preferences are multidimensional and complex (6). Ostensibly, helping patients become informed and empowered in their decision making is a laudable goal; however, it is conceivable that patients may struggle to interpret and use clinical-level performance information. Moreover, patient preferences for how to use clinician outcomes data (e.g., direct to consumer, through third-party referral sources) have received little attention.
This exploratory, multimethod study examined MHC patients’ attitudes toward, valuation of, and potential concerns about clinician selection and use of track record information about clinician outcomes in treatment decision making. For this study, a team of MHC stakeholders developed a quantitative patient survey and a semistructured interview protocol. Such efforts can inform the development and implementation of measurement-based care strategies to improve MHC decision making and outcomes.

Methods

All procedures were approved by a university institutional review board. Survey recruitment was based on rolling, voluntary self-selection between November 9, 2015, and November 1, 2016. Survey completers could then volunteer to participate in an interview to gather more in-depth and personally relevant information and recommendations. All participants were eligible for gift card compensation.
Survey participants were patients receiving treatment at one of 12 community mental health centers in the northeast United States (N=403). Inclusion criteria were any individual who was seeking or receiving MHC services and was responsible for MHC decision making for him- or herself, a family member, or an important other who was unable (e.g., because of cognitive impairments) to participate on his or her own behalf.
The study survey was developed in collaboration with a stakeholder advisory board. (A description of the stakeholder advisory board and the survey items are available as an online supplement to this report.) The survey assessed participants’ demographic information, MHC service history and provider experience, and attitudes and preferences regarding provider performance information, both in general (yes/no items) and in comparison with alternative treatment factors (Kuder Richardson-20 internal consistency = 0.74).
Through convenience sampling, 37 participants volunteered to complete follow-up interviews via conference call. Recorded interviews were transcribed. Using standards for consensual qualitative research (CQR) (7), we randomly selected 15 interviews for analysis. The final set of questions (see online supplement) covered the following domains: experiences finding an MHC provider, information that would have been helpful in making a more informed provider selection, experiences with PRO measures, interest in and reasons for (or against) accessing clinician outcome information, factors that are important when finding a provider, willingness to use and comfort level using a list of “matched” providers based on PRO information, and perceived costs or pitfalls of using provider track record information. Interview transcripts and raw audio recordings were analyzed with CQR (7) methods (see online supplement).

Results

The survey sample was mostly female (N=268, 67%) and White (N=206, 51%); the mean±SD age was 41.20±12.58 years. (A table with additional sample characteristics can be found in the online supplement.) The most common problem areas were anxiety, depression, and trauma. The most common current treatment was individual psychotherapy, followed by medication, group psychotherapy, and community-based support. The most common referral source was self, followed by another MHC provider and primary care physician. (A table with additional treatment and referral details is available in the online supplement.)
Survey results, as a percentage of “yes” endorsements for each yes/no item, are reported in Table 1. A majority of participants endorsed experiencing difficulty finding an MHC provider in their lifetime; a minority endorsed receiving a recommendation to see a specific MHC provider. When a specific referral had been provided, little explanation was given regarding its basis. Overall, participants endorsed struggling to navigate the clinician selection process and feeling as though they had received little systematic guidance. Additionally, approximately 21% of participants endorsed the belief that all MHC providers are capable of helping.
TABLE 1. Mental health care patients (N=403) endorsing survey items related to clinician choice and access to clinician outcome track recordsa
QuestionN%
15. Were there any times in your life when you wanted a mental health provider and could not find one?23458
16. Has it been hard to find a mental health provider who you were confident could help you?28170
17. Has a health care provider or agency ever recommended a specific mental health care provider to you?14035
17a. If you responded “Yes” to question 17, did the person or agency that gave you the recommendation explain what the recommendation was based on?31979
18. Has any professional ever discussed with you the pros and cons of choosing one mental health provider vs. another?9323
19. Have you ever used a consumer satisfaction rating website, such as Angie’s List or Healthgrades, to find a mental health care provider?5514
19a. If you responded “Yes” to question 19, did you find the website helpful in finding a provider?13233
20. Do you believe that all mental health care providers are capable of helping you?8321
21. Imagine that you could see a list of mental health providers’ track records in helping people with issues like your own (that is, a list of the percentage of people who they have helped versus the percentage of people who they have not helped). Would you trust these data and how they were collected?31478
22. Imagine that you could see a list of mental health providers’ track records in helping people with issues like your own. Would you use this list to help you select your provider?36791
23. Imagine that a health care professional like your primary care doctor is giving you a referral for a mental health provider. Would you feel more confident about your options if you knew that this person had reviewed providers’ track records in helping people like you?37794
24. Imagine that your insurance company is giving you a referral for a mental health provider. Would you feel more confident about your options if you knew that your company had reviewed providers’ track records in helping people like you?30275
25. Would you pay more out of pocket to see a mental health care provider who is listed as highly effective in treating the problems that you have?22355
26. Should mental health care consumers have access to information on the track records of providers in the local area?37894
27. Would it be important for you to be assigned or referred to a mental health care provider based on their track record in helping people with issues like your own?37593
28. Would it be more important than usual for you to be assigned or referred to a mental health care provider based on their track record in helping people with issues like your own if you previously have not benefited from mental health treatment for that problem?36591
29. Do you think access to information on the track records of mental health care providers would increase the likelihood of someone being helped by treatment?37794
30. Do you think that matching a consumer with a provider who has a track record of helping people with similar issues would increase the likelihood of that consumer being helped by treatment?37994
a
The entire survey is available as an online supplement to this report.
Remaining survey questions posed the hypothetical scenario of patients being able to access clinician track record information in the context of PRO monitoring. A majority of participants endorsed interest in accessing clinician performance track record information and the belief that such information would lead to more effective referrals and care outcomes. Additionally, over 90% endorsed the belief that matching informed by providers’ track records would increase the likelihood of being helped.
Interview coding domains and tables showing complete results are available in the online supplement, and themes are summarized here. Regarding experience with MHC, several participants remarked about struggling with clinician turnover, which retriggered the difficult process of finding a provider. Several patients reported experiencing limited clinician choice and becoming savvy in advocating for themselves as patients only after years in the MHC system. Regarding experience with PROs, most participants reported completing self-report evaluations of some kind; however, few were able to articulate the purpose of completing such evaluations. Concerning the use of provider performance information, participants expressed interest in accessing a list of well-matched providers. Several noted that they would be willing to remain on a waitlist for a longer period to see an empirically better-matched provider.
Regarding potential pitfalls of using clinician performance track records, participants expressed concern about placing too much weight on them in decision making, because patient perspectives of patient outcomes (which drive the track records) are subjective. Concerning preferences, a majority of participants identified interpersonal and relational factors as being similarly important (in relation to outcome track records) when finding a good-fitting clinician. Access to track record information was perceived as valuable, yet relational fit remained important. Participants reported that outcome track records would be valued and used to some degree, if made available. Ideally, other preference-related information would also be taken into consideration (e.g., being assigned to a clinician with a track record of positive outcomes who also matches a gender preference).

Discussion

Clinicians are an important source of outcome variability in MHC. Agencies and policy makers have advocated for quality monitoring and the use of quality data in MHC stakeholder decision making. Little is known about patients’ attitudes regarding the use of clinician outcome track records in decision making (8). These results demonstrate that patients are given little information or direction regarding clinician selection. When patients engage in MHC, unsystematic and nonpersonalized referrals may result in a poor patient-provider match and poor outcomes (9). Clinician track record information would be a valued resource in the decision-making process. Few participants endorsed the belief that all MHC providers are capable of helping. This finding may indicate that most participants do not view care providers as interchangeable or that patients lack faith in MHC in general.
Consistent with IOM recommendations, results indicate that access to and use of clinician outcome track record information would be a patient-centered MHC practice. This practice does not necessarily require directly providing track record information to patients; rather, such information could be provided to referral sources, such as primary care, to guide decisions. Many people with psychiatric problems have them treated solely in primary care (10). Yet, primary care providers often feel underequipped to meet patients’ MHC needs (11), reinforcing the importance of effective referral from primary care to MHC. Regardless of indirect or direct access, a majority of patients endorsed the belief that such information would enhance the referral process and treatment outcomes.
Results from the interviews provide more nuance. Specifically, track record information based on PRO data should not be the sole factor in referral and assignment decisions. Rather, track record information should be integrated into a broader assessment of and responsiveness to care preferences when possible. Notably, patients were somewhat protective of clinicians with regard to the implications of track record information. Both the survey and interview distinguished between patient satisfaction ratings (e.g., number of stars out of five on a website) and outcome data based on psychometrically valid self-report measures of symptoms and functioning. Participants viewed satisfaction ratings and testimonials as somewhat useful yet also expressed skepticism regarding their validity. Some expressed concern about overreliance on outcome information for personnel selection and reimbursement policies; potential negative effects (e.g., inequitable reimbursement practices due to failure to account for contextual factors) of overreliance on PROs to inform health service decisions cannot be ignored (12). However, overall, PRO-derived performance information was perceived as more valid than satisfaction ratings. Rather than rely on clinicians’ self-identified competence domains and patient testimonials, patients may value the inclusion of outcome-based track record information in provider directory platforms.
The results of this study must be considered in light of several limitations. These results come from a single study of community patients from one region of the United States. Although survey items were developed on the basis of stakeholder input with an emphasis on content validity and internal consistency was adequate, structural and predictive validity were not established. In addition, both aspects of the study were based on self-selecting, volunteer samples. Estimates of sampling error could not be calculated; thus, results may be subject to multiple sources of error.
We also recognize that care decisions and provider selection are not solely within the control of the individual. Numerous factors may limit choice, and barriers often hinder access to any service. Additionally, the framing of this study and both the survey and interview questions implied somewhat hypothetical scenarios in which patients have more freedom to make choices about their care than they might have in real-world situations, although the results clearly indicate that patients would value being more empowered in their treatment decisions. Relatedly, our sample comprised participants with relatively fewer financial resources. Although we acknowledge the implications of this characteristic for generalizability, it could be argued that this subpopulation is in particular need of decision support tools to avoid perpetuating health disparities and further marginalization.

Conclusions

Historically, MHC stakeholders have been unaware of clinicians’ outcome track records, which represents a gap in knowledge transfer. Suboptimal improvement rates in MHC may partly be due to the provider and a lack of personalized referrals or assignments. Because such treatment decisions are often not personalized or are based on convenience or the provider’s self-defined expertise (which research shows is often overestimated or inaccurate) (13), patients are equally as likely to see a provider who is stably ineffective at treating their condition as they are to see a more effective provider. Conversely, there is potential in harnessing performance information to match patients to clinicians on the basis of scientific outcomes data. Overall, patients appear to be interested in such an approach. It will be important to replicate these findings and to continue to explore potential implementation strategies.

Supplementary Material

File (appi.ps.202000366.ds001.pdf)

References

1.
Johns RG, Barkham M, Kellet S, et al: A systematic review of therapist effects: a critical narrative update and refinement to review. Clin Psychol Rev 2019; 67:78–93
2.
Saxon D, Barkham M: Patterns of therapist variability: therapist effects and the contribution of patient severity and risk. J Consult Clin Psychol 2012; 80:535–546
3.
Kraus DR, Bentley JH, Alexander PC, et al: Predicting therapist effectiveness from their own practice-based evidence. J Consult Clin Psychol 2016; 84:473–483
4.
Wright CV, Goodheart C, Bard D, et al: Promoting measurement-based care and quality measure development: the APA mental and behavioral health registry initiative. Psychol Serv 2020; 17: 262–270
5.
Rewarding Provider Performance: Aligning Incentives in Medicare. Washington, DC, National Academies Press, 2007
6.
Boswell JF, Constantino MJ, Oswald JM, et al: Mental health care consumers’ relative valuing of clinician performance information. J Consult Clin Psychol 2018; 86:301–308
7.
Hill CE: Consensual Qualitative Research: A Practical Resource for Investigating Social Science Phenomena. Washington, DC, American Psychological Association, 2012
8.
Boswell JF, Kraus DR, Constantino MJ, et al: Implications of the therapist effect for routine practice, research, and training; in Therapist Effects: Toward Understanding How and Why Some Therapists Are Better Than Others. Edited by Castonguay LG, Hill CE. Washington, DC, American Psychological Association, 2017
9.
Muir HJ, Coyne AE, Morrison NR, et al: Ethical implications of routine outcomes monitoring for patients, psychotherapists, and mental health care systems. Psychotherapy 2019; 56:459–469
10.
Cunningham PJ: Beyond parity: primary care physicians’ perspectives on access to mental health care. Health Aff 2009; 28:w490–w501
11.
Pidano AE, Dale LP, Morgan CD, et al: Different mental health-related symptoms, different decisions: a survey of pediatric primary care providers. J Child Fam Stud 2017; 26:3525–3534
12.
Wolpert M: Uses and abuses of patient reported outcome measures (PROMs): potential iatrogenic impact of PROMs implementation and how it can be mitigated. Adm Policy Ment Health 2014; 41:141–145
13.
Walfish S, McAlister B, O’Donnell P, et al: An investigation of self-assessment bias in mental health providers. Psychol Rep 2012; 110:639–644

Information & Authors

Information

Published In

Go to Psychiatric Services
Go to Psychiatric Services
Psychiatric Services
Pages: 452 - 456
PubMed: 33530733

History

Received: 20 May 2020
Revision received: 30 July 2020
Accepted: 11 August 2020
Published online: 3 February 2021
Published in print: April 01, 2021

Keywords

  1. Psychotherapy
  2. quality of care
  3. mental health care
  4. mental health clinicians
  5. patient preferences
  6. decision making
  7. treatment outcome

Authors

Details

James F. Boswell, Ph.D. [email protected]
Department of Psychology, University at Albany, State University of New York, Albany (Boswell, Oswald, Bugatti); Department of Psychological and Brain Sciences, University of Massachusetts, Amherst (Constantino, Coyne, Goodwin); Department of Psychology, Westfield State University, Westfield, Massachusetts (Morrison)
Michael J. Constantino, Ph.D.
Department of Psychology, University at Albany, State University of New York, Albany (Boswell, Oswald, Bugatti); Department of Psychological and Brain Sciences, University of Massachusetts, Amherst (Constantino, Coyne, Goodwin); Department of Psychology, Westfield State University, Westfield, Massachusetts (Morrison)
Jennifer M. Oswald, Ph.D.
Department of Psychology, University at Albany, State University of New York, Albany (Boswell, Oswald, Bugatti); Department of Psychological and Brain Sciences, University of Massachusetts, Amherst (Constantino, Coyne, Goodwin); Department of Psychology, Westfield State University, Westfield, Massachusetts (Morrison)
Matteo Bugatti, Ph.D.
Department of Psychology, University at Albany, State University of New York, Albany (Boswell, Oswald, Bugatti); Department of Psychological and Brain Sciences, University of Massachusetts, Amherst (Constantino, Coyne, Goodwin); Department of Psychology, Westfield State University, Westfield, Massachusetts (Morrison)
Alice E. Coyne, M.S.
Department of Psychology, University at Albany, State University of New York, Albany (Boswell, Oswald, Bugatti); Department of Psychological and Brain Sciences, University of Massachusetts, Amherst (Constantino, Coyne, Goodwin); Department of Psychology, Westfield State University, Westfield, Massachusetts (Morrison)
Brien Goodwin, M.S.
Department of Psychology, University at Albany, State University of New York, Albany (Boswell, Oswald, Bugatti); Department of Psychological and Brain Sciences, University of Massachusetts, Amherst (Constantino, Coyne, Goodwin); Department of Psychology, Westfield State University, Westfield, Massachusetts (Morrison)
Nicholas Morrison, Ph.D.
Department of Psychology, University at Albany, State University of New York, Albany (Boswell, Oswald, Bugatti); Department of Psychological and Brain Sciences, University of Massachusetts, Amherst (Constantino, Coyne, Goodwin); Department of Psychology, Westfield State University, Westfield, Massachusetts (Morrison)

Notes

Send correspondence to Dr. Boswell ([email protected]).
This study was presented in part at the International Society for Psychotherapy Research Conference, July 3–6, 2019, Buenos Aires.

Competing Interests

The authors report no financial relationships with commercial interests.

Funding Information

This research was supported by a grant from the Robert Wood Johnson Foundation (73049).

Metrics & Citations

Metrics

Citations

Export Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

For more information or tips please see 'Downloading to a citation manager' in the Help menu.

Format
Citation style
Style
Copy to clipboard

View Options

View options

PDF/EPUB

View PDF/EPUB

Get Access

Login options

Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.

Personal login Institutional Login Open Athens login
Purchase Options

Purchase this article to access the full text.

PPV Articles - Psychiatric Services

PPV Articles - Psychiatric Services

Not a subscriber?

Subscribe Now / Learn More

PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.

Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).

Media

Figures

Other

Tables

Share

Share

Share article link

Share