Best Practices: Using Case Vignettes to Train Clinicians and Utilization Reviewers to Make Level-of-Care Decisions
Abstract
Note from the column editor: Dr. Rosenquist and his colleagues describe how their academically based health maintenance organization joined in training for level-of-care decision making with the external managed behavioral health organization that was providing utilization review and case management decisions. The academic department later took over its own utilization review and in so doing internalized the utilization review function.This development, which is beginning to occur in several states, is an important solution to the "assault" that many providers of care have experienced as a result of the utilization review process. Having taken this step to deal with the realities of 21st-century health care, the authors then seize the opportunity to use their own data to improve decision making within the clinic. This process is how we get to best practices.
Medical necessity has emerged as the de facto standard for decisions about payment for behavioral health services, despite criticisms leveled from theoretical (1,2) and practical (3) perspectives. Moreover, it has been difficult to define best practices within the current framework of medical necessity, largely because of the many sources of variation in decisions about appropriate levels of care.
In practice, when clinicians and clinical case managers—that is, utilization reviewers—communicate information about a patient, usually by telephone, they use a narrative case presentation. Discussion is focused on assessment of necessity criteria, such as symptom severity, dangerousness, social support, and resource availability, that would support a higher level of care, such as inpatient hospitalization. Unfortunately, the dialogue may break down without resolution of differences, and with considerable residual ill will between parties (4).
In this column we report on our use of case vignettes as a training device to help clinicians and clinical case managers make consistent decisions about appropriate levels of care and to develop best practices.
Development and use of case vignettes
Partly because of their research and teaching missions, academic centers have been slow to react to changes in health care financing and have a reputation of inefficiency and overuse of intensive levels of care (5). In 1995 Wake Forest University established a health maintenance organization with about 50,000 enrollees. In the first year of operation, the university contracted with an outside, for-profit managed behavioral health organization to provide behavioral health utilization review and case management. Eventually the department was able to establish its own internal managed behavioral health care organization (6).
Initially, however, the department struggled to meet the demands of managed care, working with the outside organization. To promote greater uniformity in decision making, we designed four case vignettes and used them in joint training with our clinicians and the clinical case managers from the outside organization.
Case vignettes have been used previously to compare decision-making strategies of different groups (7). Because use of vignettes limits variation in how people perceive a case by providing all persons with the same information, vignettes offer training advantages over real-life patients. Case vignettes are ideal when the primary objective is to identify conflicts in judgment (8).
Each vignette developed by the department is a typical narrative case presentation and includes details about the patient's history and mental status. Each patient has a different diagnosis—delirium, comorbid depression and substance abuse, chronic depression, and schizophrenia. In two vignettes, the patient presents as an outpatient. In another, the setting is an emergency room. In the fourth case, the patient has been referred for a consultation to an inpatient general medical setting. The vignettes do not convey any expectations about case disposition or information about insurance status.
Thirty-one persons participated in the training—seven attending physicians, 16 house officers (psychiatric residents), and eight clinical case managers. After reading each vignette, respondents were prompted to choose the most appropriate treatment setting—inpatient care, partial hospitalization, or outpatient care. They provided up to five of their own reasons for each decision. In a translational process paralleling the interaction between clinician and reviewer, the reasons were examined to determine whether they referred to either of two common medical-necessity criteria: the patient's level of dangerousness and the patient's support system. These reasons were tallied separately and compared by group using Fisher's exact tests.
Results are summarized in Table 1. All the respondents recommended that the patient with delirium receive inpatient care. Similarly, for the patient with comorbid depression and substance abuse, 94 percent recommended inpatient care, and only 6 percent recommended partial hospitalization.
For the patient with schizophrenia who was experiencing negative symptoms, 52 percent of respondents recommended outpatient services, 36 percent recommended partial hospitalization, and 13 percent recommended inpatient care. For the patient with chronic depression, 39 percent recommended outpatient services, 26 percent recommended partial hospitalization, and 36 percent recommended inpatient care.
No significant differences were found between attending physicians and clinical case managers on any of the four level-of-care decisions. Only house officers selected inpatient treatment for the patient with schizophrenia. Compared with attending physicians and case managers, house officers selected higher levels of care for this patient; however, the difference was not significant.
Compared with attending physicians and case managers, house officers were significantly less likely to take into account the patient's support system as a factor in decision making in two cases—the patient with schizophrenia (Fisher's exact test, p=.09) and the patient with chronic depression (p=.09). Attending physicians were less likely than house officers and case managers to take into account the patient's level of dangerousness in their decision about the patient with chronic depression (Fisher's exact test, p=.06).
Discussion
Our study failed to demonstrate significant differences between groups of clinicians and utilization reviewers in level-of-care decisions for any of four common psychiatric presentations. Similarly, we identified very few differences between groups in their use of particular criteria as a rationale for their decisions.
Both the clinicians and the utilization reviewers in our setting have expressed surprise at these findings, because they run counter to the expectation that level-of-care decisions and decision rules used by each group would be quite different. In a study of implementation of a managed care plan during the course of which use of inpatient services markedly declined, it was shown that over time clinical case managers rated fewer patients as severely disturbed while ratings by clinicians remained unchanged (8).
Use of the vignettes has effectively demystified the process of utilization review for clinicians in our department. Two vignettes engendered strong agreement by all respondents. The others revealed more variation in decisions about the most appropriate level of care, both across all respondents and within respondent groups. This finding raises the question of how we can increase the level of agreement for more equivocal cases. Level-of-care decisions must be reliable—that is, care managers must make similar decisions in similar cases across time. Without some degree of reliability, a meaningful best practice is unlikely to emerge.
First, we must develop meaningful and reliable criteria. Some progress has been made. In one study, when clinicians were presented with a broad and unstructured list of variables, they were unable to achieve an acceptable level of agreement about indicators for hospitalization (9). On the other hand, expert panels using modified Delphi techniques have achieved high levels of agreement in decisions about levels of care for both hypothetical and actual cases, and in the process they have identified and developed anchored ratings for a number of key variables (10,11).
Second, level-of-care decision criteria must not remain the sole province of health services researchers and clinical case managers. Instead, they should be widely disseminated to the network of providers. At our facility, the outside managed behavioral health care organization shared its criteria with clinicians from the outset. When the department took over care management, this practice was continued. Also, medical-necessity criteria have been incorporated into the admission forms, admitting orders, and computerized treatment planning documentation of the inpatient and partial hospital unit (12).
Experience and training would seem to be likely sources of variation in decision making in clinical and managed care settings. Although our study was limited by its small sample size, the results suggest that house officers may differ from more experienced psychiatrists and clinical case managers in their decisions and approach. By ensuring that the house officers encounter the decision criteria in the course of their daily work and by providing them with training material on managed care principles, we hope to create a working model of best practices against which they can compare their decisions. Senior residents may also participate in an elective rotation in managed care during which they can review cases and make interpretations of medical necessity.
Criterion-based admission policies and procedures clearly narrow the range of variables used in level-of-care decisions. However, we will continue to encounter equivocal cases. One approach to improving the reliability of decisions would be to conduct field tests to systematically identify sources of variation in decision making. Once we know the sources, we may more clearly define what constitutes best practice. In a study using videotaped interviews conducted in an emergency room, agreement between raters was low for recommended disposition, psychopathology, impulse control problems, ability to care for self, and danger to self (13). A somewhat higher level of agreement was reached for psychosis and substance abuse.
Clinical case managers and medical directors continue to oversee care management in our system through traditional review processes. Every three months appeals are presented for discussion and comment before a quality improvement committee composed of a rotating group of network clinicians. Level-of-care criteria are reviewed and amended annually. This body recently voted to begin using the Criteria for Short-Term Treatment of Acute Psychiatric Illness, jointly published by the American Academy of Child and Adolescent Psychiatry and the American Psychiatric Association (15).
As practice guidelines become more detailed, and more reflective of best practices, we anticipate an eventual eclipse of more generic level-of-care criteria. For example, the use of the Clinical Institute Withdrawal Assessment protocol (16) in our facility has supplanted the need for concurrent review of necessity and intensity of service for alcohol detoxification because such a review is part of the protocol. The measure of our success will be how well we work collectively to meet the needs of patients as we develop our mental maps, whether they are vignettes, criteria, practice guidelines, or protocols.
Footnote
Dr. Rosenquist is assistant professor and Dr. Kramer is associate professor in the department of psychiatry and behavioral medicine at Wake Forest University School of Medicine in Winston-Salem, North Carolina. Ms. Briggs is chief executive officer of Carolina Behavioral Health Alliance in Winston-Salem. Dr. Colenda is professor in the department of psychiatry at Michigan State University in East Lansing. Dr. Lancaster is regional medical director of Value Options in Raleigh, North Carolina. Send correspondence to Dr. Rosenquist at the Department of Psychiatry and Behavioral Medicine, Medical Center Boulevard, Winston-Salem, North Carolina 27157-1087 (e-mail, [email protected]). William M. Glazer, M.D., is editor of this column.
References
1.
Asch D, Hershey J: Why some health policies don't make sense at the bedside. Annals of Internal Medicine 122:846-850, 1999
2.
Glassman PA, Model KE, Kahan JP, et al: The role of medical necessity and cost-effectiveness in making medical decisions. Annals of Internal Medicine 126:152-156, 1997
3.
Ford WE: Medical necessity: its impact in managed mental health care. Psychiatric Services 49:183-184, 1998
4.
Strom-Gottfried K: Applying a conflict resolution framework to disputes in managed care. Social Work 43:393-401, 1998
5.
Carey RM, Engelhard CL: Academic medicine meets managed care: a high-impact collision. Academic Medicine 71:839-845, 1996
6.
Reifler B, Briggs J, Rosenquist P, et al: A managed behavioral health organization operated by an academic psychiatry department. Psychiatric Services 51:1273-1277, 2000
7.
Jones TV, Gerrity MS, Earp J: Written case simulations: do they predict physicians' behavior? Journal of Clinical Epidemiology 43:805-815, 1990
8.
Elstein AS, Kleinmuntz B, Rabinowitz M: Diagnostic reasoning of high- and low-domain-knowledge clinicians: a reanalysis. Medical Decision Making 13:21-29, 1993
9.
Thompson JW, Burns BJ, Boldman HH, et al: Initial level of care and clinical status in a managed mental health program. Hospital and Community Psychiatry 45:599-603, 1992
10.
Hendryx MS, Rohland BM: Psychiatric hospitalization decision making by CMHC staff. Community Mental Health Journal 33:63-73, 1997
11.
Strauss G, Chassin M, Lock J: Can experts agree on when to hospitalize adolescents? Journal of the American Academy of Child and Adolescent Psychiatry 34:418-424, 1995
12.
Glazer WM, Gray GV: Psychometric properties of a decision-support tool for the era of managed care. Journal of Mental Health Administration 23:226-233, 1996
13.
Rosenquist PB, Colenda CC, Briggs JB, et al: Riding a Trojan horse: computerized treatment planning using managed care principles. Managed Care Quarterly 4:1-7, 1996
14.
Way BB, Allen MH, Mumpower JL, et al: Interrater agreement among psychiatrists in psychiatric emergency settings. American Journal of Psychiatry 155:1423-1428, 1998
15.
American Academy of Child and Adolescent Psychiatry and the American Psychiatric Association: Criteria for Short-Term Treatment of Acute Psychiatric Illness. Washington, DC, American Psychiatric Press, 1997
16.
Sullivan JT, Sykora K, Scheiderman J, et al: Assessment of alcohol withdrawal: the revised Clinical Institute Withdrawal Assessment for Alcohol scale (CIWA-Ar). British Journal of Addiction 84:1353-1357, 1989
Information & Authors
Information
Published In
History
Published online: 1 November 2000
Published in print: November 2000
Authors
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBGet Access
Login options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).