The competence of persons with Alzheimer's disease and other cognitive disorders to give consent to participate in research studies continues to raise ethical concerns, including calls for formal competency assessments of potential research participants (
1). But performing a full, formal capacity evaluation for every potential subject can be time-consuming and expensive. Clinicians and researchers may want to know whether a familiar, widely used test—the results of which are usually already available—could inform them of the likely results of a full capacity examination.
The Mini Mental State Examination (MMSE) (
2) is widely taught and used among health care professionals, serving as a universal, if limited (
3), indicator of severity of impairment among persons with Alzheimer's disease and other cognitive disorders. Several researchers have explored the relationship between MMSE scores and consent capacity, but the results have been mixed (
4). The studies used diagnostically heterogeneous populations, focused on treatment consent capacity, and used different criterion standards for capacity.
In this study we prospectively examined the limits and the utility of the MMSE as a screening tool for assessing the capacity of patients with well-defined Alzheimer's disease to consent to participate in research studies.
Methods
Participants
Our study participants were recruited from the outpatient geriatric neurology and psychiatry clinic at the Monroe Community Hospital, an affiliate of the University of Rochester, between May 1999 and April 2000. Potential participants were persons who had a diagnosis of probable Alzheimer's disease by the criteria of the National Institute of Neurological and Communicative Disorders and Stroke and the Alzheimer's Disease and Related Disorders Associations (NINCDS-ADRDA criteria) and had received an MMSE score of 18 or above in the preceding 12 months, although at the time of the study some patients had an MMSE score below 18. The cutoff was chosen to identify participants who were capable and incapable of providing consent (
5). MMSE scores range from 0 to 30, with lower scores indicating greater impairment.
Measures
The MacArthur Competence Assessment Tool, Clinical Research version (MacCAT-CR) (
6) is a semistructured interview that measures performance on legally relevant standards for competence with 21 items, each scored on a 3-point scale from 0 to 2, indicating, respectively, inadequate, partial, or adequate. The items reflect Appelbaum and colleagues' (
7) four-abilities model of competence: the ability to understand factual information (13 items), the ability to appreciate and apply the facts to one's own situation (three items), the ability to reason and manipulate the facts appropriately (four items), and the ability to communicate a choice (one item). In this report, only the data on the first three more substantive standards are presented.
The interview portrayed a hypothetical, randomized, double-blind, placebo-controlled clinical trial for a new medication for Alzheimer's disease. The MacCAT-CR has been shown to have excellent content validity and good inter-interviewer and interscorer reliabilities (
7,
8,
9). The version used in this study can be obtained from the first author.
We categorized participants as either capable or incapable on each of the three standards on the basis of validated cutoff scores of 18 of 26 for understanding, 5 of 6 for appreciation, and 6 of 8 for reasoning (
8). Because there is no consensus yet on which legal standard to use in the research consent setting, a conservative categorization method of combining all three standards was added to the three individual standards by which participants were categorized as incapable if they failed any of the three individual standards. Thus each subject was categorized as capable or incapable according to four different criterion standards. The MMSE was administered on the same day as the capacity interview.
The study was approved by the research subjects review board of the University of Rochester, and written informed consent was obtained from the participants or their surrogates.
Statistical analysis
We performed a receiver operating characteristic (ROC) analysis with SPSS 10.0. This analysis plots the sensitivity of a test against the value of 1 minus specificity, where sensitivity is the ability of a test to identify a true case (a true positive) among all cases and specificity is the ability to identify a true noncase (a true negative) among all noncases. In this study, a true positive was defined as an incapable participant, as determined by one of the four criterion standards, who scored at or below a given MMSE score. The 95 percent confidence intervals for areas under the ROC curves were calculated.
Results
The study group comprised 37 persons. Sixty percent were women. The mean±SD age was 78.7±5.8 years, the mean number of years of education was 13.8±3.2, and the mean MMSE score was 22.9±3.8 (range, 16 to 28).
Table 1 lists the areas under the ROC curves and their 95 percent confidence intervals for the MMSE as a test for identifying incapacity against the three criterion standards established by the MacCAT-CR or the combined standard. None of the confidence intervals include .5—the expected area if the null hypothesis were true—indicating that the MMSE significantly adds to the identification of incapacity. However, this significant effect is modest: the areas are relatively small, and no cutoff point yields both high sensitivity and high specificity.
High specificity, in the range of 85 to 94 percent, was approached at MMSE scores of 19 or less. Such a cutoff score correctly identified 39 to 54 percent of the incapable subjects. High sensitivity, ranging from 91 to 100 percent, was approached at an MMSE score of 26 while a specificity of 29 to 36 percent was retained.
Discussion and conclusions
Past attempts to define the utility of the MMSE in capacity assessments have focused on treatment consent in heterogeneous patient groups, with mixed results (
4). Our study is the first to assess the screening utility of the MMSE in a group with well-defined Alzheimer's disease, using a validated capacity instrument specifically designed for research on informed consent, the MacCAT-CR, as a benchmark.
Within a certain range of scores, the MMSE was at best a modest discriminator of decisional capacity in this study group. This range, 21 to 25, includes the cutoffs often used for "normal" for the MMSE. Given that abilities relevant to competence are correlated with executive functions (
10), the modest discriminatory power of the MMSE may reflect the instrument's relative insensitivity in detecting executive dysfunction.
Do these findings imply that the MMSE has no role in screening for incapacity? The utility of a test depends partly on its purpose. For instance, for a nontherapeutic, greater-than-minimal-risk study, a researcher may have strong ethical reasons to select only patients who are clearly capable of providing consent. Such a researcher would be advised to use a higher MMSE cutoff score—for example, a cutoff of 26 has a sensitivity of 91 to 100 percent in identifying persons who are incapable of providing consent—as a prelude to a more comprehensive task-specific evaluation. Alternatively, if the goal were to identify as many capable subjects as possible, even at considerable expense of effort and time, then a cutoff of 19, which has a specificity range of 85 to 94 percent, might be used. Anyone who scores below that level is very likely to be deemed incapable of providing consent.
Thus, although the MMSE is not a very good predictor of incapacity, our findings suggest how best to extract the modest amount of utility the instrument offers—with virtually no additional effort, because the scores are already available for most patients with Alzheimer's disease.
Our study is limited by its small sample, the use of a hypothetical research scenario to assess capacity, and its time-slice view of complex phenomena that may fluctuate over time. Competency judgments also take into account important contextual issues, such as the risk-benefit calculus (
1). In this study, we used a scenario for a placebo-controlled, double-blind, randomized trial for a new treatment for Alzheimer's disease. Thresholds for competence cannot be assumed to be equal across research studies with different risk-benefit ratios. Finally, because sensitivity and specificity are features of tests and not of populations, the true usefulness of the MMSE as a screening device will depend on the prior probability of incapacity in a particular situation.
Acknowledgments
This research was supported in part by grants MH-18911 and MH-64172 from the National Institute of Mental Health and by the American Association for Geriatric Psychiatry Eisai-Pfizer Alzheimer's Disease Research Fellowship awarded to Dr. Kim. The authors thank Colleen McCallum, M.S.W., Pierre Tariot, M.D., and Yeates Conwell, M.D.