Using Patient-Reported Outcomes in Schizophrenia: The Scottish Schizophrenia Outcomes Study

A prospective naturalistic observational design was utilized, which is described in detail in the SSOS study report published by NHS Quality Improvement Scotland ( 13 ). Ethics committee approval from the Scottish Mulitcentre Research Ethics Committee was obtained for the study, and written informed consent was obtained.

Participants were recruited from all NHS Health Boards in Scotland and from a number of national advocacy organizations. Approximately 9,000 people with schizophrenia were known to be in contact with NHS mental health services in Scotland ( 14 ), and we aimed to recruit a 10% sample of this population. Patients with an ICD-10 F20–F29 diagnosis (schizophrenia, schizotypal disorders, or delusional disorders) ( 15 ) were identified by key worker clinicians from their caseloads, from which a random sample was invited to participate. A total of 748 clinicians participated: nurses (N=620, 83%), occupational therapists (N=74, 10%), social workers (N=37, 5%), and psychiatrists (N=17, 2%). Clinicians received standardized training in the use of the outcome measurement tools and training on how to use the information obtained in order to continuously improve practice at individual, team, and service levels. HoNOS training was accredited by the Royal College of Psychiatrists.

The core measures used in SSOS were HoNOS ( 16 ) and the Avon Mental Health Measure (Avon) ( 17 ). The validity and reliability of HoNOS and Avon have been established among patients with severe mental health problems ( 7, 16 ). HoNOS was designed for use by U.K. clinicians, and it is now used in Europe, Australia and Canada. It comprises 12 items rated on a severity scale of 0 to 4. Possible scores range from 0 to 48, with higher scores indicating a higher severity of mental health problems. The 12 rated items can further be categorized into the following four subscales: behavior problems (aggression, self-injury, and substance use), impairment problems (cognitive dysfunction and physical disabilities), symptomatic problems (depression, hallucinations and delusions, and other psychological problems), and social problems (personal relationships, overall functioning, residential and living conditions, and occupation and activities). HoNOS was designed for use in routine clinical practice as a record of a patient's progress. A HoNOS rating was made by clinicians before collecting the completed Avon, which was self-rated by patients.

Avon is a patient-reported needs assessment tool designed by service users and health professionals in the United Kingdom for use by service users. The scale comprises five categories with a total of 29 items rated on a 5-point severity scale. Possible scores range from 0 to 145, with higher scores indicating less need over five categories. The categories (and items) are physical (food, accommodation, physical health, self-care, and ill effects of treatment), social (support, discrimination, daily routine, community involvement, and participation), behavior (sleep, risk to self, substance misuse, suicide, and anger), access (transport, use of transport, information availability, information understanding, communication, income, and managing money), and mental health (mood swings, depression, unusual thoughts, anxiety, obsessive thinking, and forgetting). In this study Avon was completed by patients separately from their clinician to ensure that their responses on Avon reflected their needs. When patients needed help to complete the assessment, this was provided by caregiver, friends, or advocacy workers. The completed assessment was then collected by clinicians.

The study began in 2002 and concluded in 2005. Data were collected during three time periods (phases 1, 2, and 3), where phase 1 represented baseline and phases 2 and 3 occurring at approximately 12 and 24 months after baseline, respectively. The data collection form ( 13 ) was brief to allow accurate completion within the time constraints of normal clinical practice. Completed data forms were then forwarded to the study center in Glasgow, Scotland, where data were verified and entered in an Access database.

Statistical analysis was conducted by using SAS 9.1 for Windows. Baseline data were tabulated, and trends over time in outcomes, treatments, and services—including psychiatric hospitalization, detention, imprisonment, enrollment in the Care Programme Approach (CPA), attempted suicide, and self-harm (all in the previous 12 months)—were estimated by using statistical models. (The CPA is a U.K.-wide system of care for patients with complex needs.) For binary outcomes a mixed-effects logistic regression model was used, with subject as a random effect and time as a fixed categorical effect (phases 1, 2, and 3) to estimate the odds ratio [OR] (with 95% confidence interval [CI] and associated p value) for phase 2 versus phase 1 and phase 3 versus phase 1. An autoregressive correlation structure across the years was assumed, and a generalized estimating equations approach was used to fit the model. Phase 2 and 3 were compared by using a likelihood ratio test. Finally, a number of covariates were introduced that might be predictive of the outcome in question to adjust the estimated effect over time. The values of covariates as measured at each phase were used rather than those at baseline. ORs for covariates that were jointly significant at p<.001 are selectively reported. The covariates utilized are described in the footnote to Table 2. For the continuous outcomes (Avon and HoNOS), a similar modeling strategy was used—a standard repeated-measures linear model.

The ICD-10 diagnoses of 1,015 participants consisted of the following: F20.0–F20.9 (schizophrenia) (N=945, 93%), F21.0–F24.0 (schizotypal disorder, persistent delusional disorders, acute and transient psychotic disorders, and induced delusional disorder) (N=20, 2%), F25.0–F25.9 (schizoaffective disorder) (N=50, 5%). Of the 1,015 persons recruited, 91% (N=919) completed phase 2, and 78% of the cohort (N=789) completed all three phases. Of the total cohort 710 (70%) were male. A total of 995 (98%) were white, seven (1%) were Asian, three (<1%) were Chinese, one (<1%) was black African, and nine (1%) did not list their race or ethnicity. A total of 781 (77%) were outpatients. The mean±SD age was 43±11 years (males, 42±11 years; females, 44±11 years; range 18 to 78 years). A total of 579 (57%) were diagnosed ten or more years ago, and 133 (13%) had a comorbid diagnosis of a substance misuse disorder. A total of 916 (90%) were registered with a primary care physician. A total of 862 (85%) received treatment voluntarily, 42 (4%) were detained in a hospital under criminal detention procedures, and 107 (11%) were detained in a hospital under civil detention (Mental Health [Care and Treatment] [Scotland] Act 2003). A total of 493 (49%) were living alone, and 318 (31%) were living with a caregiver. A total of 725 (71%) were living at home, with 188 (19%) a resident in the NHS (that is, living in long-term rehabilitation units) and 102 (10%) living in a supported accommodation. Only three were employed in a competitive job; 18 (2%) were in a sheltered work program, and 29 (3%) worked as a volunteer.

A total of 595 (59%) had no psychiatric admission to a hospital during the year before recruitment, 224 (22%) had one admission, 65 (6%) had two admissions, and 37 (4%) had three or more. Of these admissions, 202 (20%) were legal detentions, and 21 individuals (2%) had recently been in prison. A total of 187 (18%) were enrolled in the CPA. In the year before recruitment, 54 (5%) attempted suicide and 90 (9%) had self-harmed.

Table 1 shows the marginal proportions over the three phases of selected objective outcomes, and Table 2 gives the ORs and CIs comparing phase 2 and phase 3 against phase 1 (baseline) and shows the influence of selected prognostic covariates. These tables show significant improvements across the study period in the categories of hospitalization, attempted suicide, self-harm, and civil detention. Tables 3 and 4 are analogues of Tables 1 and 2 for continuous outcomes (HoNOS and Avon).

Mean Avon scores increased significantly across the three phases (representing reported improvement), and this was true for all five subsections of Avon ( Table 3 ). Higher total Avon scores were associated with being male and taking clozapine. Self-harm and the use of discretionary antipsychotic medication were associated with lower Avon scores ( Table 4 ).

In contrast to Avon, reduction in HoNOS scores represents improvement, and across the duration of the study, total HoNOS scores and scores on the behavioral problems and symptomatic problems subscales remained unchanged, while scores on the impairment problems subscale significantly increased (indicating increased impairment). Only scores on the social subscale decreased significantly, representing improved functioning ( Tables 3 and 4 ).

The importance of clinicians' use of outcomes data in the management of their own patients was incorporated into the training and educational sessions delivered by the research team. This local emphasis is fundamental for the acceptance and engagement of clinicians in the routine use of outcomes data ( 19 ). During the course of SSOS, information on aggregated outcomes was fed back to clinicians at three national meetings, demonstrating the way in which outcomes data can be utilized to configure services to meet the needs of specific populations. When individual clinicians were using the Avon needs assessment for their patients, and not just the patients included in the study, they were asked to share their practice experiences at these meetings. At the end of the study a series of focus groups were also held with patients who had participated, and study findings were reported back to them. Their views on using the patient-reported needs assessment tool were also elicited and will be available in a separate report.

Patient-rated outcomes have been defined as the measurement of any aspect of a patient's health status that comes directly from the patient ( 21 ). The systematic assessment of the patient's perspective may provide valuable information that could be lost if relying only on clinical evaluation. The Food and Drug Administration has recently issued guidance for the use of patient assessments as effectiveness endpoints in clinical trials ( 5 ). This study shows that patients with schizophrenia can use Avon effectively, except perhaps a few individuals with markedly reduced insight or severe cognitive impairment. Reliable evidence exists about the robustness of the predictive value of patients' perception of their own health status ( 22, 23 ). In mental health a number of instruments have been proposed, including Outcomes of Problems of Users of Services (OPUS) ( 7 ), Avon ( 17 ), Carers and Users Expectation of Services (CUES) ( 24 ), the Camberwell Assessment of Need ( 25 ), and FACE ( 26 ). Given the increasing evidence of the advantages of patient involvement ( 7, 19, 27, 28, 29, 30 ), patient-rated assessments have the potential to improve therapeutic alliances and treatment adherence.

Over three years of SSOS, all objective outcomes improved (reduced number of hospitalizations, incidents of self-harm, and detentions through the Mental Health Act and increased number of persons receiving benefits from the CPA), consistent with Avon patient self-assessments; HoNOS scores, however, remained constant, although scores indicated impairment significantly worsened and social functioning significantly improved. One reason for this disparity may be that clinicians tend to focus on symptomatic and behavioral problems, while functioning is more important to patients. This illustrates the value of using both patient-rated and clinician-rated measures. The approach used in SSOS is consistent with recovery models that emphasize functional rather than symptomatic improvement ( 10, 11 ).

In Scotland, after this study, the NHS introduced Standards for Integrated Care Pathways in Mental Health ( 31, 32, 33 ). The standards incorporate the routine collection of outcome measures for all patients with a severe and enduring mental health condition and include patient-rated assessments, including Avon. As a result of this policy, routine recording and analysis of outcomes data will become embedded within clinical practice ( 33 ). This will encourage reflective practice for clinicians and will provide data for health care providers to utilize in the redesign of services to meet patients' needs.