Improving Psychiatric Services Through Mystery Shopping

Since Rosenhan's seminal article "On Being Sane in Insane Places" ( 1 ), there has been strong interest in uncovering the real workings of public health facilities, particularly in areas where patients may not be well served, such as mental health care. Rosenhan's study enlisted healthy individuals to simulate auditory hallucinations in order to gain admission to psychiatric hospitals as "pseudo-patients." Staff failed to detect the pseudo-patients. Conversely, when the clinicians were asked to detect nonexistent pseudo-patients, they identified large numbers of genuine patients as impostors.

Ironically, Rosenhan's use of pseudo-patients to record their experiences in psychiatric facilities has been virtually untapped as a quality improvement tool in the field of mental health. On the other hand, pseudo-patients have been used extensively in general medicine. Hinton ( 2 ) described one of the earliest examples: the use of pseudo-patients to assess homeless people's access to primary health care in London in the early 1990s. The use of pseudo-patients in medical settings has now merged with a practice used in the retail industry known as "mystery shopping."

Mystery shoppers, also known as secret shoppers, have been used in the retail industry for years to assist in market research. Similarly, the use of pseudo-patients or "mystery patients" in medical settings enables health care services to be tested by trained consumers posing as patients. The concept is gaining in popularity, spurred by the consumer movement in health care and by efforts toward patient-centered health care. Indeed, income generated from the medical mystery shopping industry doubled between 2003 and 2004, although the health care industry constitutes a small percentage of the $600 million in revenues for the mystery shopping industry as a whole ( 3 ).

Medical mystery shopping programs are geared to detect problems in the delivery of health services. They do not specifically evaluate the quality of medical care, although Rosenhan's work went well beyond customer service to comment on a perceived lack of rigor in the diagnostic process. His findings suggest that medical mystery shopping can be developed to probe specific aspects of psychiatric practice beyond patient satisfaction.

Medical mystery shopping has proved to be a highly valuable, if controversial, method to improve patient satisfaction and raise the standard of health care services. Its use allows health care organizations to understand patients' expectations, uncover discrepancies between services promised and services delivered, and apply lessons learned from other industries. Countless testimonials about the value of medical mystery shopping can be found online, in newspaper articles, and in medical journals. Benefits cited include reduced waiting times and patient turnover; better explanations of procedures and proposed treatments; improved encounters with office staff, physicians, and other health care providers; and even less stressful programming on the waiting room television. Although there is little empirical evidence for beneficial outcomes of mystery shopping in mental health settings, there is reason to believe that positive results can be achieved as they have been in general medicine.

A good start for mental health settings would be to implement medical mystery shopping in the public sector, including state psychiatric and U.S. Department of Veterans Affairs hospitals, where service historically has been problematic; however, meeting quality standards is vitally important for accreditation. Inpatient studies have shown that even in areas that have been the target of performance improvement initiatives for years, suboptimal outcomes continue to be seen, particularly at the time of hospital discharge ( 4 ). Community mental health centers and other outpatient community systems are also prime targets for medical mystery shopping, because these settings treat many patients and have less accountability than inpatient settings have. Also, mystery shopping could be used in primary care settings, where a majority of mental health patients receive most or all of their care.

Medical mystery shopping can be tailored to the specific needs of an organization or institution. For example, mystery patients can use the telephone to assess phone etiquette, hold times, and scheduling procedures and to determine the number of potential new patients or clients "lost" at the point of entry. Walk-in mystery patients can assess the interpersonal skills and efficiency of office staff and the ambiance of the facility. Full-service mystery patients can collect information to evaluate the interpersonal skills of clinicians and adherence to policies and procedures, providing feedback about the total patient experience after a clinical encounter. Mystery shopping has helped some medical practices to improve patient callback procedures, establish a more confidential sign-in policy, and institute a more professional dress code for clinicians ( 5 ). Mystery shopping programs also have been used to identify staff training needs and to reward outstanding employees. Feedback provided by mystery shoppers can be incorporated into pay-for-performance plans, but it should not be relied on as the sole source of data for evaluating clinical performance. All of these factors also appear to be equally important in the delivery of psychiatric services.

Full-service mystery shoppers typically work from a list of questions that allow them to compile a report once the encounter is complete. Sample questions might include the following:

• Can the facility be reached by public transportation, and is the facility accessible to persons with physical disabilities?

• Is the receptionist or intake coordinator cheerful and friendly?

• Is the waiting or intake area clean and comfortable?

• Are privacy and confidentiality maintained during the intake?

• Is the psychiatrist attentive, compassionate, and generous with his or her time?

• Are the risks, benefits, and side effects of prescriptions explained?

To be sure, medical mystery shopping has limitations. No program will be successful unless there is complete buy-in of the concept among all employees and they are informed up front that mystery patients will be seeking services from time to time. Employees must understand that feedback from mystery patients is important to patient satisfaction and the success of the organization. Mystery patients should be similar in demographic characteristics to the actual patient population served by the organization and use language patterns similar to those of regular patients—for example, avoiding technical terms. Optimal results are realized when mystery patients visit at regular intervals to track progress over time and to evaluate improvements that have been made regarding previously identified issues. Mystery patients should provide constructive feedback and not be used by employers for the purposes of a "sting" operation.

Even under these conditions, clinicians may resist medical mystery shopping for ethical reasons. Many physicians find it deceitful, even when it is known that mystery patients may present for treatment. A greater concern is the potential for mystery patients to interfere with or compete for resources that are needed by patients who are ill and need treatment, particularly in emergency departments. In addition, some clinicians are concerned that third parties might attempt to access information collected by mystery patients that includes personally identifying data. Finally, there is the remote possibility that mystery patients can be harmed or traumatized if given unnecessary or unwanted treatment.

These objections are valid, but they can be overcome when medical mystery shopping programs are well designed. In fact, the American Medical Association's Council on Ethical and Judicial Affairs recently endorsed medical mystery shopping as a method for promoting quality improvement, noting that "physicians have an ethical responsibility to engage in activities that contribute to continual improvements in patient care" ( 6 ). A medical mystery shopping program helps raise staff awareness of the need to serve a broad range of patients. For those with psychiatric illnesses, who may be unassertive or easily intimidated or may lack the capacity to advocate for change, medical mystery shopping may be an ideal method to improve the quality of psychiatric services.

Additional field testing and research are needed, however, to determine how medical mystery shopping compares with more traditional methods to assess patient satisfaction (such as surveys) and whether mystery shopping can reliably lead to superior outcomes compared with questionnaire-type surveys. Analyses need to be conducted to determine whether a greater yield of information obtained with medical mystery shopping compared with surveys is worth the cost. Research should also be done to shed light on possible differences between programs using mystery patients who are general consumers versus programs that employ bona fide "simulated patients"—individuals who have undergone extensive training to act as real patients in order to test specific skills of clinicians.

Despite many unanswered questions about applying the concept of mystery shopping to the psychiatric setting, one thing seems obvious. The health care industry has never been noted for its customer service, and as competition builds amid efforts to encourage patients to comparison-shop for health care, psychiatric facilities should be looking for innovative ways to improve the patient experience.