A Longitudinal Study of Learning Potential and Rehabilitation Outcome in Schizophrenia

The study took place within the framework of the research project "Vocational Rehabilitation and Integration of Persons With Mental Illness" ( 30 ), which was approved by the local research ethics committees. Study participants were recruited from a comprehensive vocational rehabilitation program for persons with chronic mental illness in Halle-Saale, Germany. The program aims to improve work skills and future vocational prospects for people with mental and social impairments through support and individual training under simulated (sheltered) work conditions ( 30 ).

The program took place in the Rehabilitation Psychisch Kranker (Institution of Rehabilitation for the Psychiatrically Ill and Disabled) in the city of Halle. The institution contains classrooms for teaching and enclosed workshops for woodwork or metal crafts. Patients initially stay in the program for nine months and can complete three to six more months if recommended by the supervisors. The patients' mean±SD length of stay in the rehabilitation program was 332±96 days.

In the first six to eight weeks (diagnostic phase), the rehabilitation team evaluated the patients to determine their current mental health status. The seven-hour (8 a.m. to 3 p.m.) daily program comprises several courses of general vocational training as well as classes in specialized subjects such as mathematics, basic computer skills, administration or bookkeeping, and metal crafts or woodwork (training and stabilization phase). Depending on individual needs, courses are offered for training in social communication and social competence, concentration and memory, as well as management of daily activity. In the course of the program, demands are increased qualitatively and quantitatively in a stepwise manner but supplemented by stress relief and relaxation periods to support and stimulate the patients. Medical and psychotherapeutic treatments as well as social support accompany the rehabilitation program.

Work training in natural settings takes place within the last three months of the program (hands-on phase), after which patients return to the rehabilitation institution for final evaluation. Eighty-two patients were admitted to the program from November 2002 to January 2004. Program participants were eligible for the study if they fulfilled the DSM-IV diagnosis of schizophrenia or schizoaffective disorder, were symptomatically stable on neuroleptic medication, had German as their native language, were between the ages of 18 and 65 years, had no co-occurring diagnosis of substance abuse or dependence according to DSM-IV, and had premorbid IQ not lower than 70 points.

Thirty-nine rehabilitation participants had mental disorders other than schizophrenia or schizoaffective disorders or mismatched other inclusion criteria. Two patients refused to participate. Thus the study sample consisted of 41 persons (16 women), all of whom completed the program and the study.

The Wisconsin Card Sorting Test (WCST). The WCST ( 31 ) is a widely used measure of concept formation and cognitive flexibility. Under standard administration, respondents are required to match 128 stimulus cards containing different geometric shapes (varying in color and number) to one of four target cards. After each response, test takers receive feedback that the response was right or wrong. Respondents are not informed of the rule for correct matching or that the rule changes after they have matched ten consecutive cards.

In this study the WCST was conducted in a pretest-posttest sequence within one session, with each block comprising 64 cards. Pretest (trial A) and posttest (trial C) were identical to the standard procedure. Between trial A and trial C, a training block (trial B) was administered in accordance with the approach of dynamic testing ( 21 ) and with the trial-by-trial interventions described by Green and colleagues ( 32 ). The sorting rules were explained to the research participants, and they were told whether and why their response was right or wrong (for example, "This was right; we're actually sorting for color" or "This was wrong; we aren't sorting for color now, but for form or number"). In addition, participants were informed of category change (for example, "With ten consecutive correct sorts now completed, the rule will change. You will no longer sort for color but for form or number").

For the analyses of intraindividual change of performance, the number of correct responses was selected on the basis of its advantageous distributional characteristics. Change was assessed with a model developed by Schoettke and colleagues ( 33 ), which has a psychometric basis, controls for ceiling and floor effects, and is appropriate for its ease of clinical application. It provides distinct categories of individual learning potential (learner status), is based on the analysis of the individual case, and was successfully validated in various studies ( 19, 22, 23, 24, 25, 26, 27 ). On the basis of the standard error of prediction, a confidence interval derived from the model that specified the range wherein deviations from a hypothetical parallel test score (on subsequent test administrations) could be considered as indicators of real change or not. The algorithm results in the following classification rule: for significant change of intraindividual performance, a cutoff of 15 points was defined, which represents approximately 1.5 standard deviations in the original study.

With a maximum of 64 cards per trial, measurable improvement cannot be expected above an initial score (trial A) of 43 correct responses. Participants were classified as high scorers when they scored 43 or higher in trials A and C. Persons who improved by at least 15 points from trial A to trial C were classified as learners, whereas those who did not improve their performance by a minimum of 15 points and did not achieve a score of 43 in trials A and C were classified as nonlearners. Study participants who scored 43 or better on trial A and less than 43 on trial C also were classified as nonlearners. Finally, a category for participants whose performance declined by at least 15 points was provided.

Control variables. For the assessment of sociodemographic data and characteristics of the illness (age at first admission, duration of the illness, and type and dosage of medication), a standardized sociobiographic interview was administered at enrollment to the program (time 1). The severity of psychiatric symptoms was assessed at each assessment point (times 1 to 4) with the Positive and Negative Syndrome Scale (PANSS) for schizophrenia ( 34, 35 ). The PANSS is a 30-item rating scale with three rationally derived subscales (positive symptoms, negative symptoms, and general symptoms).

Verbal intelligence as a measure of premorbid performance was tested at study intake (time 1) with subtests 1 and 2 of the Performance Assessment System (Leistungspruefsystem, or LPS) ( 36 ). This valid and reliable instrument (retest reliability of .95) assesses the main factors of intelligence outlined by Thurstone ( 37 ). Both subtests were administered with standardized instruction and were conducted with time limitations. The patients were required to identify misspellings in a succession of words of increasing difficulty.

Outcome measures. For the assessment of work capabilities at program intake (time 1), during rehabilitation (time 2), and at program termination (time 3), we used the Osnabruecker Arbeitsfaehigkeitenprofil (O-AFP, or Osnabrueck Ability to Work Profile) ( 38 ). The O-AFP represents a reliable translation and adaptation of the Work Personality Profile ( 39 ) for use in Germany. This profile is especially suitable for programs that most closely simulate the general work environment. It consists of 30 items scored on a 4-point scale, ranging from 1, "problem area that will definitely limit the chances for employment," to 4, "a definite strength, a plus for employment." Assessment was provided by the patient's immediate supervisor and based on the general labor market and its demands as standard. A rating of 3 to 4 means that the rehabilitation participant is able to work in the general labor market; a 2 indicates likely employment in the sheltered work environment.

Assessment of the rehabilitation participant was performed on three subscales of the O-AFP, each with ten items: learning ability, social and interactional competence, and social adaptation and motivation. Internal consistency ratings for these subscales were, respectively, α =.954, α =.909, and α =.899.

General functional outcome was measured by the Level of Functioning Scale ( 40 ). With this four-item rating scale, outcome is measured quantitatively according to four dimensions, each rated on a 5-point scale: work (from 0, none, to 4, daily work activities including studies, housework, sheltered employment, and day center activities within the past year [time 1] or since the last assessment [times 2 to 4]), symptomatology (from 0, severe, to 4, no symptoms within the past month), social relations (from 0, no social relations, to 4, meeting friends at least once a week), and hospitalization (from 0, more than 75% of the time, to 4, never within the past year [time 1] or since the last assessment [times 2 to 4]).

In accordance with Reker ( 7 ), the level of vocational integration was assessed qualitatively with an ordinal scale containing five levels: level 1, competitive employment or regular apprenticeship; level 2, work or apprenticeship under protected conditions; level 3, vocational rehabilitation for persons with mental illness; level 4, unemployment (no current employment but being available for the job market); and level 5, permanent disability payments (no current employment and not being available for the job market). This scale takes into account that, under the present economic conditions in Germany, vocational integration is often reached only partially through integration in protected work environments or continuing vocational rehabilitation for patients with severe mental illness. The level of vocational integration was assessed on the basis of the patients' level of vocational activity at the time of the follow-up assessment.

We applied a repeated-measures design with four assessment points: initial assessment at time 1 took place at the patient's admission into the rehabilitation program; time 2 assessment was conducted after 26 weeks of attendance in the program. Time 3 assessment took place at individual program termination, which was after 332.0±96.1 days' stay in the program. Time 4 assessment was conducted three months after individual program termination.

Time 1 assessment started after study participants had given their written consent. Sociodemographic data and characteristics of the illness were obtained, and diagnostic assessment was conducted with the German version of the Structured Clinical Interview for DSM-IV ( 41 ). Within the first session, psychopathology, as determined with the PANSS, was examined, and level of functioning was initially rated. During the second week of the rehabilitation program, we tested intelligence (with the LPS performance assessment) and assessed executive functioning and learning potential (with the WCST). Patients were classified into the WCST learner groups according to the classification algorithm ( 33 ).

The rating of work capability (O-AFP) was provided by the patient's immediate supervisor in a 15-minute consultation of all rehabilitation staff members working with the patient. The time 1 O-AFP evaluation took place only about four weeks after the patient's entrance into the rehabilitation program and therefore represents no true baseline. This delay occurred because some exposure to the rehabilitation environment is necessary before attaining a score on this instrument.

At times 2 and 3 psychopathology (PANSS), level of functioning and work capabilities (O-AFP) were assessed. At time 4 assessment included rating of psychopathology (PANSS), level of functioning, and the level of vocational integration or reintegration.

After completion of the WCST at time 1, study participants were classified according to their individual learner status with the algorithm ( 33 ) described above. Correlations of covariates and outcome at baseline were assessed by the Spearman correlation coefficient.

In a first evaluation step, we calculated a doubly repeated measurement model ( 42 ) to account for the two levels of correlation in our data set (first, with the repeated measurement-in-time scale, and second, by several outcome measures at each time point) simultaneously. Next, group comparisons of the single-outcome measures were evaluated with repeated-measures models, including a group factor (WCST learner status), a time factor (measures at different time points), a group × time interaction, and the PANSS measurements as covariates.

For the intraindividual correlation in the time course, we initially hypothesized four different patterns: independent, compound symmetry, first-order autoregression, and completely unstructured. For each outcome measurement model, we chose the intraindividual correlation pattern that resulted in the best model fit according to the Bayes information criterion. Group effects on the different time points were tested by post hoc tests in the respective models. As a sensitivity analysis, we performed nonparametric analyses for the four outcome measures, following the ideas of Brunner and colleagues ( 43 ). Because the results were essentially unchanged, we report only the results of the parametric analyses. Finally, differences on vocational integration between groups were assessed by nonparametric methods (Kruskal-Wallis and Wilcoxon tests). Statistics were computed with SPSS 12.0 and SAS 9.1.3.

Learning potential classification yielded 18 high scorers (44%), 16 learners (39%), and seven nonlearners (17%). No participants with a declining performance of a minimum 15 points were identified. Groups did not differ in gender, age, LPS intelligence subtests 1 and 2, education level, age at first psychiatric hospitalization, duration of illness, or duration of participation in the rehabilitation program. We found no significant group difference in the distribution of the type of schizophrenic illness, neuroleptic medication, daily dose of chlorpromazine equivalents, and PANSS psychiatric symptoms at time 1.

However, baseline negative symptoms were correlated with the Level of Functioning score (r=-.437, p=.004) and the O-AFP social adaptation and motivation subscale (r=-.384, p=.013) at time 1. PANSS general symptoms were found to be correlated with level of functioning (r=-.477, p=.002). Thus PANSS symptoms were entered as time-varying covariates in the analyses described next.

The doubly repeated measurement model resulted in a significant overall group effect for the WCST learner status (F=4.62, df=8 and 111, p<.001), a significant effect of time (F=6.25, df=9 and 342, p<.001), and a significant group × time interaction (F=1.99, df=18 and 342, p=.01). The model results for the specific outcome measures are reported next.

Work capability. Table 2 presents the O-AFP data for the learner groups at the first three assessment points. With PANSS subscales entered as time-varying covariates, a hierarchical linear model for the learning ability O-AFP subscale yielded a significant effect for the group factor (p=.004) but not for the repeated-measures factor or for the interaction. Tests of effect segments yielded a significant group effect for all assessment points (time 1, p=.006; time 2, p=.004; time 3, p=.039).

At time 1 learners did not differ from high scorers and from nonlearners, whereas high scorers differed from nonlearners (p=.002). At time 2 learners differed from high scorers (p=.024) but not from nonlearners. Again, high scorers and nonlearners differed significantly (p=.002). At time 3 learners did not differ from high scorers, but learners differed from nonlearners (p=.017), and nonlearners and high scorers still showed a significant difference (p=.011). For the O-AFP subscale of social and interactional competence, no significant group effect or effect of repeated testing was found, but there was a significant group × time interaction (p=.038). However, post hoc tests showed no significant difference between any combinations of factor levels. On the social adaptation and motivation subscale of the O-AFP, neither a significant group effect nor an effect of repeated measurement and interaction was found.

Level of functioning. General level of functioning according to the Level of Functioning score for the three learner groups and all four assessment points is presented in Figure 1 . A hierarchical linear model with PANSS subscales entered as time-varying covariates yielded a significant effect for the repeated-measures factor (p<.001) but not for the group factor and the group × time interaction. Lack of statistically significant differences over the long term did not preclude different levels of outcome in the learning potential groups after program termination. To test for this, we conducted tests of effect segments that yielded a significant group effect for time 4 (p=.002). Single comparisons for time 4 showed that learners did not differ significantly from high scorers but differed significantly from nonlearners (p=.001). High scorers significantly differed from nonlearners (p=.001).

Level of vocational integration. At time 4 groups significantly differed in their level of vocational integration (Kruskal-Wallis χ ² =8.29, df=2, p=.016): six of the high scorers (33%) and six of the learners (33%) but none of the nonlearners were competitively employed. Another six of the high scorers (33%), one of the learners (6%), and none of the nonlearners were integrated in work programs under supported conditions. No high scorer, one of the learners (6%), and one of the nonlearners (14%) took part in another rehabilitation program for persons with mental illness. Six of the high scorers (33%), eight of the learners (50%), and four of the nonlearners (57%) were unemployed. Two of the nonlearners (29%) but none of the high scorers and learners received permanent disability payments from social pension funds.

Post hoc single comparisons (Bonferroni adjustment for three comparisons from α =.05 to α =.017) revealed no difference between learners and high scorers, but there was a trend toward a difference between learners and nonlearners (Mann-Whitney U=24.50, p=.022) and a significant difference between high scorers and nonlearners (Mann-Whitney U=18.00, p=.004).