Dr. Goldman and Colleagues Reply

TO THE EDITOR: Dr. Steen questions our use of expert physician panels for testing the validity of utilization management criteria by pointing out that the InterQual criteria were developed to challenge practitioners toward more efficient resource use than is currently practiced. We have several major disagreements with Dr. Steen's letter.

First, we disagree with Dr. Steen's statement that our study defined validity as the matching of current practice. Current practice in our study was represented by the actual care provided according to the medical records studied. We did not compare the judgments of the criteria sets to a standard based on this care; instead, we compared the criteria to the consensus judgments of a panel of psychiatrists, selected by their peers as having special expertise in making judgments regarding the need for acute care. The panelists were instructed to ignore the constraints of actual practice by assuming that all treatment options existed at each Veterans Affairs facility studied and that private-sector psychiatric practice applied to the VA. The panel's consensus judgments were more stringent than the actual practice for 45% of the admissions and continued-stay days we studied; i.e., the panel decided that the patient should be treated on an ambulatory basis rather than as an inpatient. Furthermore, the panel's judgments were frequently more stringent than the assessments based on the InterQual criteria. The criteria indicated that inpatient care was needed while the panel judged that ambulatory care was appropriate for 36% of the discrepancies between the 1993 InterQual criteria and the panel and for 43% of the discrepancies between the 1992 InterQual criteria and the panel.

We also disagree with Dr. Steen's suggestion that to “push the frontier,” developers should strive for utilization criteria that are more stringent than expert clinicians' assessments based on the available scientific evidence. His view of criteria development (and, implicitly, of validity testing) is in conflict with the great bulk of the literature on the development of clinical guidelines (e.g., 1–3) and assessment criteria (e.g., 4, 5) and with all prior studies of the validity of utilization criteria of which we are aware (e.g., 6–8). This literature emphasizes expert clinician assessment of scientific evidence in the development and testing of criteria and guidelines. It is our belief that Dr. Steen's more subjective approach to criteria development places too much emphasis on cost reduction and too little on expert clinical judgment and scientific evidence relating to the needs of patients for appropriate care.

Finally, Dr. Steen indicates that the InterQual criteria, even if invalid, will do little harm because they are designed to flag cases for discussion with practitioners and review by physician advisers. However, it is clear from accounts of the utilization management process that the criteria significantly influence these processes (9, 10). Even if the criteria were used only as warning signals, it clearly would be more conducive to appropriate patient care if they were valid warning signals.