Measures of Personal Recovery: A Systematic Review

Searches were conducted by using six data sources. First, eight databases were searched from date of inception to May 2012: MEDLINE, PsycINFO, EMBASE, CINAHL, CSA Illumina, TRIP, CDSR, and DARE. The search terms were divided into four domains—personal recovery, mental illness, measure or instrument, and psychometric properties. The terms were identified from the title, abstract, key words, and medical subject headings (MeSH). The search terms were amended for each database as necessary. A copy of the full list of search terms is available from the first author.

Second, 11 Web-based repositories were searched by using the terms “personal recovery,” “mental health,” and “measure.” The repositories were the Department of Health (www.dh.gov.uk), the Scottish Recovery Network (www.scottishrecovery.net), the Centre for Mental Health (www.centreformentalhealth.co.uk), Recovery Devon (www.recoverydevon.co.uk), the Repository of Recovery Resources (www.bu.edu/cpr/repository/index.html), Mind (www.mind.org.uk), Rethink (www.rethink.org), the National Mental Health Development Unit (www.nmhdu.org.uk), the Mental Health Commission of New Zealand (www.mhc.govt.nz/), the Mental Health Commission of Ireland (www.mhcirl.ie/), and the Mental Health Commission of Canada (www.mentalhealthcommission.ca/).

Third, a search of Google Scholar (www.scholar.google.co.uk) was conducted by using the terms “personal recovery,” “mental health,” and “measure.” Fourth, abstracts from three international conference series were searched by using the terms “personal recovery” and “measure.” The conferences were biennial conferences of the European Network for Mental Health Service Evaluation (1994–2010), annual meetings of the American Psychiatric Association (1999–2009), and Refocus on Recovery conferences (2010 and 2012). Fifth, the table of contents was searched by hand for the following journals: Psychiatric Services, International Journal of Methods in Psychiatric Research, and Psychiatric Rehabilitation Journal. Sixth, reference lists of included papers were assessed for further measures.

To evaluate the extent to which the identified measures focus on aspects of recovery, four raters independently compared the measures to the CHIME framework. Each item on the measure was mapped to one of the five CHIME framework categories, if possible. Items covering more than one domain were assigned to the domain that it represented most. Items that did not map, and, therefore, did not assess personal recovery, were counted as not mapping. A concordance in allocation to CHIME category of over 70% was achieved between raters.

Each measure was evaluated for nine psychometric properties by using criteria modified from several authors (19–21). For each property, the measure was rated positive (adequate demonstration), indeterminate (inconclusive evidence), negative (inadequate demonstration), or not enough information (no evidence available).

From 336 abstracts reviewed, the search identified for inclusion 35 articles, which described 13 measures. Eight articles described measure development, nine were reports of psychometric properties, 16 articles described studies that used a measure as part of outcome assessment, and one article compared measures of personal recovery and clinical recovery. One measure, the Stages of Recovery Scale, was unobtainable and was excluded from analysis. The 12 measures are described in Table 1. [A flow diagram of the identification of the measures is available online as a data supplement to this article.]

All measures are rated by service users, and the IMR also has a clinician-rated version. The two measures cited most widely in the search were the RAS and the IMR. The RAS appeared in 13 articles, including four describing psychometric properties, eight describing use as an outcome measure (three studies each from the United States and Australia and one each from Canada and Sweden), and one comparing consumer-defined and clinically defined recovery measures. The IMR appeared in eight articles, including two describing psychometric use and six describing studies that used it as an outcome measure (one each from Sweden and Israel and four from the United States). The only other measure used as an outcome assessment, by two studies from the United States, was the Recovery Markers Questionnaire (RMQ) (31).

The mapping of items from each measure to the CHIME framework is shown in Table 2. The QPR was the only measure to have every item map to the CHIME conceptual framework and the only measure to have at least 10% of items in each category.

Three measures (the Mental Health Recovery Measure [32], the RMQ, and the Self-Identified Stage of Recovery [33]) were not included in any psychometric or scale development articles and, therefore, were excluded from psychometric analysis. Three measures were included in more than one psychometric article (the RAS, four articles, and the IMR and the STORI, two each). Findings from all articles were included in the evaluation. The psychometric evaluation of measures is shown in Table 3.

The measure with positive ratings for the highest number of the nine investigated properties was the STORI (N=6 properties), followed by the Maryland Assessment of Recovery (MARS) (34) (N=5); the QPR and the RAS (N=4); the IMR, Recovery Star (35), and the RPI (N=3); and the Short Interview to Assess Stages of Recovery (33) (N=2). The psychometric properties demonstrated by the most measures were content validity (N=9 measures), internal consistency (N=8) and test-retest reliability (N=7). The least demonstrated properties were criterion validity (N=0), responsiveness (N=1), reading age (N=1), and feasibility (N=2).

This review had three strengths. First, the methodological rigor advances the field by using more sources of data than existing reviews (21,26,29) and double ratings of eligibility as a quality check, by including a flow diagram, and by operationalizing the definition of recovery. Second, although existing reviews included staff-rated measures and measures of recovery orientation, there was substantial overlap between the measures identified in this review and earlier reviews. Third, the distribution of country of development for the measures (Table 1) supports the view that recovery conceptualizations—at least as present in English-language sources—have primarily emerged from the English-speaking world (37).

The review had three limitations. First, it excluded non–English-language papers, possibly missing some measures. Second, the term “personal recovery” is not a MeSH heading in databases, and, therefore, the search may have lacked some specificity and missed some measures. The use of the large number of search criteria aimed to counter this possibility. Finally, applying the CHIME framework is a subjective process that depends on individual interpretations of the measures and the meaning of each item. Each CHIME category contains many subcategories (12), so a measure may not adequately assess a category if only a small number of items map to it. A measure’s uneven coverage of the CHIME categories means that the summary score emphasizes different components of recovery. CHIME also represents only one conceptualization of recovery, and the use of other frameworks or conceptual backgrounds may have resulted in different findings. Several measures have items not mapping to CHIME, possibly because these items represent nonrecovery elements of best practice or because CHIME needs to be extended to incorporate other recovery processes. Although the CHIME framework captures an understanding of recovery within the English-speaking world (37), its wider applicability is unknown.

The challenge in mental health services is developing a system that is based on the views of service users about how their recovery can be supported. Problems with conceptual clarity and inadequate psychometric testing of recovery measures are barriers to meeting this challenge. Having a clear understanding of the conceptual basis of recovery is necessary to enable service providers to provide recovery-focused interventions. If a service is to be recovery focused, the interventions need to be oriented toward this goal. Whether mental health services should be judged in relation to their recovery orientation or by change in service recipients’ experience of recovery is an important scientific and clinical question. Stronger measurement tools are needed for either approach.

It is also necessary for service providers to be able to identify the resource implications of using recovery measures in routine outcome assessment. The development of consensus on how recovery is measured will allow investigation of the resource consequences of transformation toward a recovery orientation. In order for recovery measurement to advance to the next stage of scientific development, it will be necessary to refine theoretically defensible (38) and psychometrically adequate measurement tools that address recovery-specific evaluation issues (39).