Performance Measures: From Proliferation to Implementation

Sunderji and colleagues have admirably undertaken the daunting task of examining and cataloguing, in detail, the vastly diverse universe of integration outcome performance measures published to date. The most sobering finding of their work is the extensive variability and complete lack of standardization in both what is being measured and how it is being measured. Of 1,255 implementation and outcome measures, the investigators identified 148 “unique” measures, yielding an average of 8.5 published variants of each unique measurement. Clearly, creating measures and publishing them is much easier and quicker than implementing them, let alone systematically using them. In fact, only 67% (841 of 1,255) of the published measures have been implemented. It is difficult to see the value of publishing a measure that is never employed. Of the implemented measures, the authors identified 385 that they deem useful. If the previous average of 8.5 variants for each “unique” measure holds up for the subpopulation of measures, that still leaves a total of 45 potential performance measures for integrated care. I’ve seen very few organizations that can systematically measure and improve more than six to eight performance measures at once.

We have become lost in a sea of highly variable minutia. It is like trying to apply performance measures to snowflake production. Everything is unique, and we have no consensus on what is optimal. We need to step back and find consensus on basic principles and then adhere to them as we pursue the particulars.

The first and most important goal of measuring performance is not to improve performance, but rather to determine the range of performance and where we, as individual practitioners or organizations, fall within that range. We were all taught that we have a duty to practice within the usual accepted community “standard of practice,” but how are we to know what the standard of practice is? We were commonly taught that it is a given truth derived from a textbook or lecture. More recently, numerous publications proclaim the consensus of various groups of experts on the standard of practice, in rather detailed terms. But surely, some part of the usual community standards of practice must include the range of current practices. There is very little published information on this, and as the Sunderji et al. article amply shows, there is a remarkable lack of uniformity or agreement on how to measure the range of practice. We can effectively undertake projects and interventions to improve our performance only after we learn what the range of outcomes is and where each of us falls within that range. Working on performance improvement without knowing the baseline of the range and one’s position in it is like measuring progress to a destination when you are lost.

Standardization is the basis for continuous improvement and quality. It is not possible to improve things that are not standardized because without standardization, we cannot, for example, be sure that we’re giving the same treatment or program twice in a row, let alone whether the treatment or program from one clinician or organization is substantially the same as that from another. Ensuring uniformity of a treatment or program over time or across different treatment settings requires process measurement. Process performance must be defined before outcomes can be effectively measured across multiple providers, let alone improved.

Setting uniform standards for a small, operationally manageable set of measures could be the responsibility of providers, payers, or government. In the case of health care, all three have failed to systematically push for standardization because it would mean accepting compromises and constraints on themselves and, more important, entail a great deal of painful work. The providers of health care—integrated, behavioral, or other—are so diverse and numerous that there is no hope of a push for standardization from their side. That leaves the payers and government. In most other mature industries, such as automotive, aviation, retail, and food, major purchasers are deeply involved in improving the quality of operations of their suppliers. The pay-for-performance implementation efforts of payers and government will never achieve substantial success by trying to improve dozens of “unique measures” of integrated care, each of which has multiple variants, in integrated behavioral health care or in any other area of health care.