Performance Measures for Schizophrenia Research

To the Editor: In developing of a set of performance measures for assessing schizophrenia treatment services, Addington and colleagues (1) have undertaken an important task and put a truly impressive effort into it. A more systematic approach to understanding a broader stakeholder assessment of care is much needed and timely, nowhere more than in schizophrenia. Schizophrenia has a range of fluctuating effects on patients, families, and society, and efforts to identify a single, self-evident measure for evaluating the effectiveness of services have not met with success.

The Delphi approach is an excellent technique for developing consensus and ranking consensus opinions and priorities. We have used it in a number of situations and have found it invaluable to understand treatment priorities (2,3) and outcome measures (4,5). It is, however, just a tool and should not be treated with too much respect. The instructions given must be detailed but also clear and precise, tailored to both the aim of the exercise and to the specific group engaged.

A clear understanding of the ultimate goal of the exercise is key. Although a Delphi process can be used solely for measuring consensus, users of the process are usually seeking consensus on judgments or priorities. This needs to be made clear, and the following statement is an attempt to do so: “Our aim is to identify the two most important performance measures in each domain. To achieve this, the whole range of the assessment ratings must be used.” Academics are generally comfortable giving a low rating to items, but clinicians and nonprofessionals need strong encouragement to do so. Many service user and caregiver advocacy groups struggle with assigning a low rating to any item and need constant encouragement to rank the importance of service elements. In our previous work, we had to keep reminding service users and caregivers that rating everything “important” or “essential” effectively wasted their vote: “If you rate everything important, then your ratings cancel each other out and you have no influence on the outcome.”

Addington and colleagues' Delphi process reduced 97 items to 36 that participants identified as essential. But this hardly moves us on, especially when one of the eight domains (appropriateness) ended up with 31 performance measures. I do not see how I could change my clinical practice on this basis—and promoting change in clinical practices must surely must be the ultimate purpose of the exercise. Addington and colleagues have got this far, and I would encourage them to return to the group to further refine the results. That way they would produce a focused consensus statement on schizophrenia care performance measures that nobody could ignore.