Views of Mental Health Care Consumers on Public Reporting of Information on Provider Performance

From October 2006 through January 2007, we conducted four focus groups in separate Pennsylvania Medicaid regions with 41 Medicaid-enrolled mental health care consumers and their family members. We used multiple strategies to recruit participants, including letters and phone calls directly to individuals, indirect outreach via county personnel and individual treatment facility administrators, and on-site recruitment on the day of each focus group.

An experienced facilitator conducted 90-minute focus groups. The focus groups initially discussed current uses of provider performance information after examples were provided of commonly used performance measures (such as the Healthcare Effectiveness Data and Information Set measures regarding seven-day and 30-day follow-up care after discharge and optimal practitioner contact after initial prescribing of medications) and commonly used formats for presenting information, such as report cards. Participants were then asked to discuss what types of provider performance information would be most useful and relevant, how the value and relevance of the information could be increased, and what ways of presenting information would make it most accessible and useful. Probes were used to clarify responses. The University of Pittsburgh Institutional Review Board approved the study, and participants gave verbal consent.

Focus group discussions were recorded, transcribed, and reviewed by research team members to explore general themes that arose. Focus group process notes provided additional context about the discussions. Two research team members independently coded focus group transcripts for major domains of inquiry based on an initial transcript review. When coders assigned different categories to the same text, a discussion was held in research team meetings until consensus was reached about how statements should be coded. Categories were provisional and finalized after a process of constant comparison in which each statement was checked against similar data and against a more inclusive category that described statements in a class. Subsequently, the research team discussed the content of each domain and refined the coding schemes by expanding, collapsing, or eliminating codes until there was consensus.

Several key themes emerged across focus groups, including the ease of accessing information and the type of information that consumers consider to be most useful. However, participant discussion about the utility and formats of commonly used performance measures (such as those provided as examples) was conspicuous by its absence. As one participant remarked when discussing the issue, "I don't care if [a particular provider delivers excellent care] … since I can't get there. It's just going to make me mad." Highlighted below are key themes that emerged and examples from the focus group discussions.

A consistent theme was the desire to have publicly reported provider information that was easily accessible and updated frequently. Participants observed that no particular, optimal process exists to disseminate this information and that varied dissemination approaches were likely to reach more individuals. One consumer explained, "I'd like [to receive] a booklet on services available in my local area that I can access," and other participants said that they wanted information to be available at the provider site: "Send [a book or a pamphlet] to places where mental health [care] is provided." The importance of frequent updates was also commonly discussed. Observing frequent changes to programs and personnel in community mental health organizations, one participant offered, "It has to be updated; you can't have [information] not being current."

Many participants discussed the desire to know more about provider services, allowing for more informed choices about care with that provider. For example, many wanted information about the clinical expertise available. As one consumer said, "I'd like to know what [the clinicians'] level of expertise is and what areas they've had training in. Autism, ADHD, drug and alcohol issues, things like that. What sort of credentials? [Knowing whether] somebody has lots of experience versus somebody who's new." Other participants were interested in the range of services available: "Be aware of [different services] because you are not even aware of them until someone has brought it up. So making people aware of what services are available for each provider." Other participants were interested in times of availability, "So we can look at availability versus flexibility. Say if a doctor or therapist is only in their office two days a week or all week, or Monday, Tuesday, Wednesday they are there and the rest of the week they have evening hours."

Equally important to consumers, however, was knowing whether they would be able to use this information in making choices at provider organizations—could they choose their clinicians and the types of services they might want? "[Knowing if we will be able to] see who's on a list and choose who [we] would like to see [is important]. They just assigned this new [clinician] to me without even asking" exemplified a common issue. Participants also discussed the importance of knowing the extent of shared decision making at a provider organization. As one remarked, "Because I have to do [treatment] their way. That's very upsetting to me. I think … in the idea of recovery, [choosing treatment approaches] … should be a lot more flexible. And to hear that, after all the trainings I've been to, it's extremely upsetting, that somebody feels that they have to conform to the old way of doing things."

Participants across all groups emphasized the importance of knowing whether consumers would receive care in a timely manner. Many discussed this in the context of prior unsatisfactory experiences, such as the participant who said, "What about … how long it takes to get an appointment? And do they call you back? A lot of times you get a machine and no one ever calls you back … and then, is it a three-month wait to get in?" The importance of timely care was reflected by one participant's observation, "Because when you want the help, usually it's immediate, you know. Here and now." Yet this participant, like most, realized the importance of reasonable expectations. "A week to ten days is a reasonable amount of time to look at scheduling an appointment. Not November, when you're calling in September or August." Many participants also observed that a clinic's ability to provide timely care was also influenced by consumers' and families' behavior, reflected by the woman who observed, "Parents and people who use the services need to be held accountable, too. Because I hear repeatedly, 'I couldn't get an appointment,' but … there were five cancellations that day. People called to cancel their appointments or just didn't show up, and they could've seen my son."

Knowing about provider flexibility and responsiveness in scheduling appointments was another frequent theme. One participant described receiving a letter that indicated, "'This is your appointment date and time.' And 'if that date and time doesn't work,' then it was, 'we'll send you a new letter with a new date and time.'" Another participant recommended, "You should [know if you] have an option [for scheduling appointments], because what if you got [one] that didn't work out for your family's schedule?" Others described being frustrated with a lack of responsiveness, contrasting it with general medical care providers, "They give you an appointment. And depending on symptoms … it's almost like you're forced to go to Emergency. And going [there] should be the last resort, not a first resort. If I had a problem with my chemo, they wouldn't say, 'Well, check with your radiologist in a month.'"

The ease of conversing directly with a psychiatrist was another aspect of performance that participants frequently discussed. As one participant remarked, "[You want to know that if you call you're going to be able] … to talk to your doctor when you feel the need to talk. Not that person who's screening the doctor's calls." Another participant wanted to know that if she called and wanted to speak with the doctor "then they have a time when they are taking phone calls or following up on those phone calls, not relaying it through the receptionist to the nurse, to the doctor, and back." Continuing, she said, "because if I call and say I want to talk to the doctor, I really don't want to talk to the nurse. I want to talk to the doctor. There needs to be some way to get to the doctor without having to go through that." Focus group participants recognized psychiatrist time constraints but described how the lack of direct communication between patients and prescribing physicians could result in poorer care.

Participants also discussed the importance of knowing more about psychiatrists' communication with their other treatment team members and health care providers. One participant described trying to discuss medication interactions with his or her primary care practitioner and being told, "Talk to your therapist. And [that person] says, 'Well, make an appointment with your psychiatrist.' It's like they pass the [buck]." Another discussed how good communications could prevent unnecessary use of services, "A lot of people complain about [mental health care consumers] using the emergency room and costs going up. If you don't want that, there needs to be a continuum of care where everything is linked … the primary care practitioner and the therapist and the psychiatrist."

Focus group participants strongly supported recent trends toward enhancing availability of information about mental health providers to consumers and families. Information about mental health treatment options and providers is becoming increasingly available to the public ( 5 ), although much of this information comes through informal sources, such as privately run Web sites, which raises concerns about potential biases and lack of objectivity ( 6 ). Our findings suggest that mental health care consumers welcome additional information about providers from formal sources, such as health care insurers or county- or state-level oversight groups. The findings are consistent with broader studies suggesting that individuals are generally positive about public reporting of information about health care providers ( 7, 8 ). Such information can empower mental health care consumers and is responsive to calls for a more consumer- and family-directed mental health system ( 3 ).

Our results suggest, however, that consumers are interested in a broader range of information about providers than is often available. Similar to what is commonly reported for other health care areas, much of the publicly available information about providers consists of indicators about the quality of behavioral health care processes ( 9 ). We found a lack of enthusiasm among consumers for many such process indicators, however, and a substantial desire to easily access more information about services offered by mental health provider organizations. This included structural quality indicators, such as clinician training and expertise, and increased access to care via appointments available during evening or weekend hours. Such indicators of quality are often more transparent to individuals than aspects of quality reflected in process indicators. This finding is consistent with studies of general health care that have found that publicly reported information needs to be more relevant and easier to understand for it to be useful ( 10 ) and that socioeconomically vulnerable and other vulnerable populations have the greatest difficulties understanding and using publicly reported data ( 11 ).

Participants wanted to know whether they would be involved with choices regarding their clinician and care. Involvement in such choices is a key component of recovery-oriented treatment systems and is central to mental health system transformation ( 12 ). Increased information facilitates shared decision making by mental health care consumers and has been shown to improve patient satisfaction and clinical outcomes ( 13, 14 ). The consumer-provided information needed to assess a provider's approach to shared decision making is highly desired by consumers but would require more effort than many currently used quality measures ( 4 ).

Increasing dissemination of information about health care providers is an important goal that receives insufficient attention ( 15 ), consistent with our finding that individuals wanted easier access to information through a range of sources. Although widespread dissemination of information would reflect a change from current practice, it is consistent with goals of increased health care system transparency and accountability.

Our focus groups comprised publicly insured mental health care consumers and family members from Pennsylvania. Results may not be representative of privately insured or uninsured individuals or individuals from different regions of the country. Our participants also were not commonly familiar with provider or health plan report cards for ranking performance, and focus group discussions may have been different if they had included individuals more familiar with their use.

Despite these limitations, our findings provide valuable information regarding desires of mental health care consumers for enhanced information about mental health provider organizations. With increased focus on provider transparency, information about provider price and quality of process is becoming increasingly available to health care consumers. It is unclear, however, to what extent information on price influences choice of providers among individuals, such as those with Medicaid coverage, for whom there is no cost sharing for services. Our findings suggest that process indicators commonly used to assess quality may be of limited utility to some mental health care consumers. Instead, our participants indicated a desire for information that describes how to access what type of provider, at what location, in what time frame, at what time, with what ability to coordinate care among treating providers. Providing this information will require routine and structured collection of information from other mental health care consumers, automated and streamlined methods of obtaining structural data from providers, and development of distribution methods that are easily accessible to consumers.

Further research is required to better understand the effects on consumer and provider behavior of gathering and making available such information. Nonetheless, efforts to increase availability of information desired by mental health care consumers is consistent with efforts to increase transparency in health care and an important step in supporting the recovery of persons with mental illness.

Support for this study was provided by Community Care Behavioral Health Organization. The authors express their gratitude to Karen Celedonia, Shari Hutchison, Samantha Shugarman, and Emily Magee for research assistance; to Kathleen McMasters, Kathy Erndl, and Lori Fertall for assistance in assembling and conducting the focus groups; to James Schuster, Mike Jeffrey, and Lydia Singley for feedback on the manuscript; and to the consumers, providers, and other members of Community Care's Member Advisory and Performance Management Committees for their thoughtful suggestions and comments on this project.

The authors report no competing interests.

Dr. Stein and Dr. Kogan are affiliated with the Department of Psychiatry, University of Pittsburgh, and with the Department of Research, Evaluation, and Outcomes, Community Care Behavioral Health Organization, Pittsburgh. Dr. Essock is with the Departments of Mental Health Services and Policy Research, College of Physicians and Surgeons, Columbia University, and the New York State Psychiatric Institute, New York City. Ms. Fudurich is with the Quality Department, Community Care Behavioral Health Organization. Send correspondence to Dr. Stein, Department of Psychiatry, University of Pittsburgh, One Chatham Center, Suite 700, Pittsburgh, PA 15260 (e-mail: [email protected].)