Impact of Modifying Risk Adjustment Models on Rankings of Access to Care in the VA Mental Health Report Card

OBJECTIVE: Using a measure from the Veterans Administration (VA) mental health report card, the authors examined stability of report card rankings under different risk adjustment models. The study determined the impact of adding one variable to a standard VA risk adjustment model for access to care and examined whether veterans who reside in counties with VA facilities are more likely than veterans who reside in counties without VA facilities to use VA mental health services. METHODS: The authors compared the proportion of service users among veterans residing in counties either with or without VA facilities; the data were risk-adjusted by using demographic and geographic variables. A variable representing the presence or absence of a VA facility in each county was added to the risk adjustment model, and its impact on regional VA report card rankings for access was calculated. RESULTS: Regional rankings for access to mental health services by all veterans changed substantially when the additional variable was introduced into the risk adjustment model. Seven of 22 regions changed by five or more places. Access to mental health services was higher in counties with VA facilities, even after geographic distance to VA and non-VA hospitals was controlled for. CONCLUSIONS: Caution should be exercised when evaluating performance on the basis of report card rankings alone, even after risk adjustment. The results emphasize the importance of attending to observed and adjusted scores, along with standard deviations and standardized scores, rather than to rank order alone and highlight the need to educate report card users in their interpretation and limitations. County boundaries should be taken into consideration when planning VA mental health services.