Barriers to the Appropriate Clinical Use of Medications That Improve the Cognitive Deficits of Schizophrenia

Neuropsychological constructs of cognition are not the same as everyday notions of thinking, planning, and reasoning. The skills assessed in neurocognitive tests are narrower than everyday tasks that people perform in the course of a day. In trail-making tests, participants connect numbers and letters in alternating order (1-A-2-B-3-C, up to 13-M). Other tests measure the speed with which respondents locate target symbols or name colors ( 23 ). Memory and working memory tests include assessment of word learning or recollection of digits or other material for brief spans. Tests of visual learning and memory assess recognition of faces or family scenes or ask respondents to reproduce a line drawing. Other tests, such as the Wisconsin Card Sorting Test, assess a person's ability to flexibly respond to changing cues in a sorting task. Tests of social cognition may ask respondents to rate the intensity of emotion expressed on an actor's face. The respondent's effort affects performance on neurocognitive tests ( 24 ).

Few data are available to clarify how clinical behaviors relate to the specific skills assessed on neurocognitive tests ( 25 ). Many neuropsychological tests have moderate ecological validity compared with everyday cognitive functioning ( 26, 27 ), but much of the data on ecological validity are gathered from patients with moderate to severe brain injuries, whose cognitive deficits are more severe than in schizophrenia. In a study of neurological patients and a control group, measures of executive function were significantly predictive of relatives' or caregivers' reports of dysexecutive problems in everyday life ( 28 ), yet a replication study showed modest correlations between these measures ( 29 ). When tests of memory were compared with clinicians' reports of day-to-day memory function in adults with brain injury ( 30 ), correspondence varied according to the severity of memory impairment and the memory test used. More ecologically valid neurocognitive assessment strategies, such as highly engaging tests that assess metacognitive capacities like self-knowledge ( 25 ), are in development.

Limitations in ecological validity data do not indicate that cognition in everyday life is entirely discrepant from that measured in the laboratory, but it does mean that clinicians may not know whether patients' behaviors reflect laboratory-identified deficits that should be treated with a cognition-enhancing medication. Individuals with schizophrenia show an impairment in their ability to make social judgments from facial expressions ( 31 ) and have deficits in the recognition of familiar faces ( 32 ) in computerized tests of social cognition. Yet these deficits do not lead to obvious difficulties in patients' ability to read a speaker's emotion or recognize faces ( 33 ). In fact, patients with paranoia may be particularly adept at detecting subtle emotional clues that validate delusional thinking ( 34 ). Problems getting to appointments or remembering medications may be caused by multiple noncognitive factors, such as lack of social supports or funds, poor insight, or behavioral disorganization.

Efficacy trials for registration of promising compounds will generate robust data about changes in laboratory constructs ( 13, 35, 36 ). The seven domains of cognition included in the MATRICS cognitive outcome battery (working memory, attention and vigilance, verbal learning and memory, visual memory and learning, speed of processing, reasoning and problem solving, and social cognition) ( 37 ) were not chosen to represent clinically obvious aspects of cognition, nor do they necessarily mark the cognitive problems that most trouble people with schizophrenia. And although cognition is usually discussed as a general capacity ( 4 ), the relationship between separable domains of cognition ( 23 ) and general aptitude is an area of controversy in the field.

Cognitive deficits are not accurately assessed through patient self-report or "clinical eye"; neither matches the results of laboratory tests. In one study ( 38 ), clinicians judged memory to be normal in 20% to 40% of psychiatric patients despite substantial impairment on objective tests. Clinical judgment differed from objective measures of other domains 33% to 53% of the time. In another study ( 39 ), clinicians showed moderately good sensitivity in differentiating patients with schizophrenia and intact cognition from those with impaired cognition, but specificity rates were low. Forty percent of persons with schizophrenia who showed impaired attention and memory on neuropsychological tests did not perceive that they had a cognitive problem, and patients were less accurate in perceiving that their cognitive skills were intact. Clinicians and patients agreed at the rate of chance. Patients' cognitive complaints do not match objective scores. Prouteau and colleagues ( 40 ) found that complaints of poor attention were associated with poor visual memory and planning. Finally, patients' reports on a brief rating scale (the Schizophrenia Cognition Rating Scale, or SCoRS, and the Clinical Global Impression of Cognition in Schizophrenia, or CGI-CogS) were not identical to interviewers' or caregivers' reports, nor to other measures of cognition or functioning ( 41 ; Ventura J, Cienfuegos A, Boxer O, et al., unpublished manuscript, 2006). The discrepancy is not limited to schizophrenia; cognitive complaints in other patient populations do not strictly correlate with objective measures ( 42, 43, 44, 45 ).

The subtlety and specificity of the cognitive deficits identified on neurocognitive tests are difficult to describe in clinical terms. These deficits are less pronounced than those of dementia, developmental delay, or mental retardation, and their upper and lower boundaries are indistinct. A small subgroup of patients with schizophrenia perform in the neuropsychologically normal range ( 46, 47, 48, 49 ). In a meta-analysis, 78% of patients with schizophrenia scored below the median on a test of global verbal memory, but only 61% scored below the median on a test of block design ( 4 ). (The median is defined as the score within a distribution where 50% of individuals score above and 50% score below.) Finally, heterogeneity has been demonstrated in individual's cognitive profiles ( 50 ), including during a first episode of psychosis ( 51 ), and many individuals with schizophrenia retain intellectual skills ( 52, 53 ).

Clinicians will need to measure cognitive skills, and brief rating scales are available for doing so. The Cognitive Screening Instrument for Schizophrenia, a ten-minute test, is being developed and validated (Purdon S, personal communication, 2006). Rating scales include the CGI-CogS, a 30-minute interview-rated scale that gathers input from patient and caregiver ( 54 ); the Brief Cognitive Assessment ( 55 ), a 15-minute scale used by interviewers; and the SCoRS, which is a ten-minute clinician-rated scale that measures patients' and caregivers' impressions of cognitive performance (Ventura et al., unpublished manuscript, 2006). Briefer versions of the cognitive battery used in clinical trials include the Brief Assessment of Cognition in Schizophrenia, which assesses several important domains of cognition in schizophrenia and takes 35 minutes to complete and score ( 56 ). The Repeatable Battery for the Assessment of Neuropsychological Status is also a brief cognitive battery that does not assess the most important domains of cognition in schizophrenia ( 12 ) but may be more suited to the severe cognitive deficits seen in dementia ( 57, 58 ). Data describing scales' correlations with the objective measures of cognition, patients' complaints, clinicians' impressions, and real-world functioning are available for some rating scales ( 41, 55 ).

Although these scales are reliable and take only minutes to administer, efforts to encourage clinicians to rely on rating scales in clinical decision making are not usually successful ( 59, 60 ). Clinicians use clinical judgment to guide most routine clinical decisions ( 61 ). In addition, education about cognition may present clinicians with ambiguous messages about the role of rating scales. Continuing medical education materials on cognition in schizophrenia emphasize that cognitive dysfunction is present "in varying degrees (often severe) in the majority of patients with schizophrenia'' ( 62 ) and that all domains are affected. Cognition and functioning tend to be conflated in these materials ( 63 ). These messages suggest that all patients with schizophrenia would benefit from a cognition-enhancing medication. In fact, whether medications will be beneficial to all patients, whether global cognitive performance will improve, and whether functioning will change are important empirical questions. Colloquial usages of "cognition" and "functioning" may erroneously imply that there would be no need to measure cognitive deficits with more psychometric precision.

Finally, the lack of agreement between clinicians, patients, and laboratory measures on the treatment target may make it difficult for prescribers to appropriately identify patients who need cognitive enhancers. Successful treatment depends on three-way agreement between families, clinicians, and patients. Clinicians may not be able to detect changes in cognition caused by the medications, or they may perceive cognitive changes that cannot be verified on tests. Patients' reports of improvement or deterioration may not be substantiated with rating scales, and their motivation for adherence may be reduced. Clinicians may also have difficulty estimating the value of cognition medications against costs, side effects, and drug interactions. Overuse of medications through inadequate treatment targeting or underuse of medications for patients who could benefit from them ( 64, 65 ) are possible outcomes.

The fundamental reason to prescribe cognition-enhancing medications is to improve patients' daily lives. It is hoped that cognitive enhancement will lead to improvements in such outcomes as overall well-being, employment, activities of daily living, and social engagement ( 66 ). Yet most of the data on cognition and functioning are correlational, associating a discrete aspect of cognition with a discrete aspect of functioning ( 67 ). Scores on tests of vigilance, for instance, have been correlated with social problem solving ( 68 ) and with independent living ( 69 ). Associative visual memory has been correlated with autonomy in activities of daily living ( 70 ). Visual memory and working memory have been shown to explain variance in work functioning ( 11 ). The effect sizes of the relationships between cognitive domains and functional constructs usually fall in the modest or medium range ( 25 ). The association between neurocognition and quality of life appears mixed ( 70, 71, 72, 73 ). It is not clear whether it is most accurate to view specific domains of cognition as correlates of specific aspects of functioning or to view the two as more generally interrelated ( 74 ).

Multiple mediating factors link cognition to real-world functioning. Negative symptoms, specific cognitive deficits, environmental factors, age, and illness duration all make varying contributions to patients' community and social functioning ( 9, 75 ). For instance, working memory and negative symptoms explained most of the variance in daily problem solving in one study ( 76 ), and verbal memory, processing speed and attention, and negative symptoms predicted outcome in another ( 9 ). Poor verbal memory predicted impairment in recreational activities, whereas attention and negative symptoms were predictive of work performance ( 9 ). Social factors, such as prior work experience, education level, social functioning, and economic circumstances, influence work outcomes ( 77 ).

The most distal, real-world outcomes that clinicians might seek to change have the least strong correlations with specific cognitive deficits. Cognitive skills predict improvement in work quality ( 78, 79, 80, 81 ) among patients in work rehabilitation, but domains of cognition have little correlation with more distal outcomes, such as type of employment ( 81, 82 ). In a study of persons with schizophrenia in India, neurocognition did not relate significantly to current employment or work performance; negative symptoms and social functioning, however, were significant ( 83 ). Yet the employment rate for those with schizophrenia in India was approximately 70% ( 84 ), compared with less than 15% in developed countries ( 85, 86 ). For some patients, local conditions and social factors may exert a much stronger effect on functioning than cognitive deficits.

Subtle shifts in functioning that are attributable to cognitive enhancers may be difficult for clinicians to detect, because improvements in functioning would occur gradually ( 10 ). The Global Assessment of Functioning scale (GAF) has sufficient validity in single evaluations ( 87 ) but insufficient reliability as a measure of change for an individual patient ( 88 ). Patients' self-reports of functioning often diverge from objective measures ( 12, 89 ). Proxy or performance-based measures have been developed for use in clinical trials ( 10, 90 ). These measures require persons to perform a simulated skill, such as paying a bill ( 91 ), preparing a meal, or responding to a simulated social interaction ( 92 ). Adequate performance on a proxy measure does not ensure that a patient can perform the skill in everyday life ( 93 ), and because proxy tests rely on contrived settings, they are not appropriate for routine clinical use.

More research on mediators between cognition and functioning may inform clinical assessment strategies. Knowledge of grocery-shopping skills has been described as a mediator between cognitive skill and grocery-shopping performance, and measures of knowledge of functional skills could be used by clinicians ( 94 ). Intermediate functional measures, such as the performance-based outcome measures, could be completed by ancillary staff, such as occupational therapists. These measures may be more strongly correlated with neuropsychological tests than brief cognitive rating scales designed for clinicians to use ( 41 ). Ecologically valid cognitive test batteries, such as those that assess metacognition, may provide clues to clinically meaningful assessment strategies. For instance, metacognition appears to be a mediator between objective cognitive deficits and poor insight ( 95 ). Recognizable clinical symptoms that can be traced directly to cognitive deficits could help clinicians direct treatments.

This article explains why attempts to accelerate therapeutic innovation can be coupled with efforts to prepare clinicians for the introduction of novel treatments. Accelerating the development of medications for cognition is a high priority, but prescribing cognition-enhancing medications effectively may be a complex clinical task. Constructs are not equivalent to daily activities or everyday notions of thinking and living, and data are not available to describe how cognitive domains correlate to the phenomena seen in the clinic. Clinicians' and patients' assessments often do not match objective assessments. Functional aspects of patients' lives are determined on multiple levels and are difficult for clinicians to detect and measure. If these barriers prevent appropriate selection of patients for treatment with cognition-enhancing medications and complicate the monitoring of responses to such medications, positive findings from efficacy trials of cognition-enhancing compounds may be difficult to translate into real-world improvements in outcomes.

Prescribing patterns for medications to treat Alzheimer's disease show the importance of addressing clinicians' informational needs ( 96 ). Specialists and internists differ widely in prescribing rates for cholinesterase inhibitors ( 97 ). Measures used in clinical trials are not directly applicable to clinical practice ( 98, 99, 100 ). Perhaps because clinicians have difficulty in assessing the efficacy of medications for Alzheimer's, both overuse and underuse have been described. Treatment guidelines are available, yet factors such as economic concerns and family preference influence prescribing. A study from Germany determined that more than 50% of clinicians prescribing cholinesterase inhibitors did not expect any positive therapeutic effect; a positive effect from the drug was expected by a minority (18%–28%) of prescribers ( 97 ). Given the similarities between the two treatment targets, cognitive enhancers for schizophrenia could be as difficult to appropriately prescribe.

Suggestions can be offered. Neuroscientific constructs of cognition and functioning ought not be conflated with everyday notions of cognition and functioning. Whereas cognition is used colloquially to indicate a general capacity to reason and process information and functioning is equated to real-world outcomes, the data are organized around discrete constructs and domains. The conflation of everyday notions with neurocognitive constructs ( 62, 63 ) may make it difficult for clinicians to use the evidence based to guide prescribing, yet the conflation could facilitate the marketing of cognition-enhancing medications ( 101 ). More data will be needed to clarify the clinical manifestations of the deficits identified in laboratory tests of cognition. Clinicians can be educated that cognitive deficits can be reliably measured but, like abnormal cardiac conduction, cannot be accurately assessed in a routine clinical interview.

The Food and Drug Administration has suggested that a compound approved as a cognitive enhancer in schizophrenia will need to improve scores on a cognitive battery and a second measure with clinical face validity ( 23, 102 ). However, these "co-primary" measures—patients' reports, clinicians' assessments, or proxy functioning—all have limitations as described above. Co-primary outcome measures in clinical trials may give clinicians useful information about cognitive drugs' clinical meaning. Regardless, it is possible that compounds that result in changes on measures used in clinical trials would have little impact on the real-world outcomes that clinicians would consider important ( 103 ), such as everyday planning and social relatedness. Measurable changes in cognition may be weakly linked to distal outcomes such as independent living or employment, particularly where such moderating factors as social networks or employment opportunities are significant. Practical clinical trials and effectiveness trials could be crucial to defining the clinical utility of cognition-enhancing medications and informing clinical decision making ( 104 ).

Several research teams are investigating the reliability and validity of clinical assessment strategies. Intermediate outcome measures, such as metacognitive scales and proxy functional measures, might be more accessible for clinicians than laboratory-based cognitive tests. Clinicians will also need better ways to assess the real-world functioning of patients with schizophrenia. More needs to be known about how clinicians should interpret patients' complaints about cognitive problems. Consumers' perspectives on cognitive interventions can be better understood, particularly regarding whether changes revealed on cognitive measures facilitate recovery and well-being ( 105 ). In summary, a range of stakeholders may provide useful perspectives on drugs in development ( 106 ) to be sure that clinical research generates the kind of data that helps prescribers direct treatments to the patients who can benefit from them.

This work was funded by the West Los Angeles Veterans Affairs Healthcare Center and the Robert Wood Johnson Clinical Scholars Program. The views expressed are those of the author and do not necessarily reflect the views of the Robert Johnson Foundation or the Department of Veterans Affairs. The author thanks Alexander Kopelowicz, M.D, and Stephen R. Marder, M.D., for helpful comments.

The author reports no competing interests.

Dr. Bromley is affiliated with the Semel Institute Health Services Research Center, University of California, Los Angeles, and with the Veterans Affairs Desert Pacific Mental Illness Research, Education, and Clinical Center. Send correspondence to Dr. Bromley, West Los Angeles Veterans Affairs Healthcare Center, 11301 Wilshire Blvd., Building 210A (MIRECC), Room 104, Los Angeles, CA 90073 (e-mail: [email protected]).