Development and Validation of a Dementia Risk Prediction Model in the General Population: An Analysis of Three Longitudinal Studies
Publication: American Journal of Psychiatry
Abstract
Objective:
Identification of individuals at high risk of dementia is essential for development of prevention strategies, but reliable tools are lacking for risk stratification in the population. The authors developed and validated a prediction model to calculate the 10-year absolute risk of developing dementia in an aging population.
Methods:
In a large, prospective population-based cohort, data were collected on demographic, clinical, neuropsychological, genetic, and neuroimaging parameters from 2,710 nondemented individuals age 60 or older, examined between 1995 and 2011. A basic and an extended model were derived to predict 10-year risk of dementia while taking into account competing risks from death due to other causes. Model performance was assessed using optimism-corrected C-statistics and calibration plots, and the models were externally validated in the Dutch population-based Epidemiological Prevention Study of Zoetermeer and in the Alzheimer’s Disease Neuroimaging Initiative cohort 1 (ADNI-1).
Results:
During a follow-up of 20,324 person-years, 181 participants developed dementia. A basic dementia risk model using age, history of stroke, subjective memory decline, and need for assistance with finances or medication yielded a C-statistic of 0.78 (95% CI=0.75, 0.81). Subsequently, an extended model incorporating the basic model and additional cognitive, genetic, and imaging predictors yielded a C-statistic of 0.86 (95% CI=0.83, 0.88). The models performed well in external validation cohorts from Europe and the United States.
Conclusions:
In community-dwelling individuals, 10-year dementia risk can be accurately predicted by combining information on readily available predictors in the primary care setting. Dementia prediction can be further improved by using data on cognitive performance, genotyping, and brain imaging. These models can be used to identify individuals at high risk of dementia in the population and are able to inform trial design.
Reliable identification of individuals at increased risk of dementia is essential for individualized risk management in both primary and specialized clinical care, but also for optimal design of preventive trials (1). This necessity was aptly demonstrated by recent findings from large randomized controlled trials that showed potential efficacy of multidomain interventions to prevent cognitive decline in high-risk individuals (2–5). The Finnish Geriatric Intervention Study to Prevent Cognitive Impairment and Disability (FINGER) trial (2) showed that a multidomain lifestyle intervention resulted in a significant protective effect on cognition. The success of this trial has been attributed in part to the tailored approach of targeting only an at-risk segment of the general population for these preventive interventions (2). This strategy was further corroborated by secondary analyses from the Prevention of Dementia by Intensive Vascular Care (PreDIVA) trial (3) demonstrating that intensive vascular risk management had the strongest effect among participants with untreated hypertension (3). It is now increasingly recognized that such preventive strategies may be most effective in an at-risk population (3, 4, 6–8).
Several models have been developed to predict dementia in the general population (9), but external validation recently showed that these have limited incremental predictive value above and beyond age (10). These models were mostly based on lifestyle factors, social factors, and comorbidities. So far, models are lacking that include information on markers that reflect the underlying disease process, especially in its early stages. Such markers include subjective memory decline, APOE genotype, and neuroimaging (11–14). On the other hand, such markers are usually not available in primary care settings. It is therefore conceivable that different models are required, depending on the setting: simple non-laboratory models for primary care settings and extended biomarker-based models for specialized clinical care settings. Note, however, that for the purposes of risk stratification in healthy individuals in primary care settings, models should preferably be based on risk factors that can be obtained without invasive diagnostics such as CSF sampling or imaging requiring substantial amounts of ionizing radiation such as positron emission tomography (PET).
Another important consideration is that dementia prediction models should take into account the competing risk of death from other causes, given the generally late-life onset of dementia among community-dwelling individuals. Failure to account for such competing risks inflates apparent dementia risk predictions, limiting the practical utility of currently available models (15).
Our aim in this study was to develop a dementia prediction model for use in primary care settings. We also examined whether an extended model that included cognitive, genetic, and imaging markers could improve predictive performance. Both models were developed while accounting for competing risks.
Methods
Study Population
This study was embedded in the Rotterdam Study, a prospective population-based cohort study (16). Since 1990, inhabitants age 55 and older residing in Ommoord, a district of Rotterdam, the Netherlands, have been invited to participate in the study. Of the 20,744 invited inhabitants, 14,926 (72%) agreed to participate. Follow-up examinations take place every 3 to 4 years. In addition, a random sample of Rotterdam Study participants were invited for brain MRI scanning in 1995 and 1996 (N=563). From 2005 onward, brain MRI became part of the core study protocol of the Rotterdam Study (17). For the present study, we selected participants age 60 or older for whom baseline data were available on clinical, cognitive, genetic, and MRI parameters (see the flowchart of study participants in Figure S1 in the online supplement ). We excluded participants who had dementia or incomplete screening for dementia at baseline (N=40), those who did not provide informed consent to access medical records (N=11), and those for whom no follow-up was available for logistical reasons (N=35). We also excluded participants for whom valid imaging data were not available because of artifacts or other reasons (e.g., contraindications to MRI or signs of claustrophobia during acquisition) (N=124) as well as those for whom data were missing on APOE status (N=134). After exclusions, a total of 2,710 participants were included for analysis in this study, all of whom provided written informed consent to participate in the study and to have their information obtained from treating physicians.
Candidate Predictors
Detailed methods on predictor data collection and predictor definitions are described in the online supplement . We prespecified candidate predictors on the basis of the literature, expert knowledge, and availability in clinical practice. For a primary care model, we considered the following candidate predictors: age, sex, education level, systolic blood pressure, smoking, history of diabetes, history of stroke, presence of depressive symptoms, parental history of dementia, presence of subjective memory decline, and need for assistance with finances or medication. For the extended model, we considered the addition of cognitive tests (word fluency test, letter digit substitution test, Stroop interference, and delayed word learning test), APOE-ε4 genotype, and brain MRI parameters (white matter hyperintensity volume, total brain volume, hippocampal volume, and presence of infarcts [lacunar/cortical]). White matter hyperintensity, total brain, and hippocampal volume were all entered into the models as a percentage of intracranial volume to correct for differences in head size.
Assessment of Dementia
Participants were screened in-person for dementia at baseline and at subsequent center visits with the Mini-Mental State Examination (MMSE) and the Geriatric Mental State Schedule organic level (18). Those with an MMSE score <26 or a Geriatric Mental State Schedule score >0 underwent further investigation and informant interview, including the Cambridge Examination for Mental Disorders of the Elderly. The information from in-person screening was supplemented by data from the electronic linkage of the study database with medical records from all general practitioners and the regional institute for outpatient mental health care. In the Dutch health care system, the entire population is entitled to primary care that is covered by their (obligatory) health insurance. The entire cohort is thus continuously monitored for detection of interval cases of dementia or cognitive disturbances between center visits. Study physicians evaluate all records biannually and combine information from medical records with in-person screening to draw up individual case reports. In these reports, the physicians covered all gathered relevant information to establish the presence, probability, and subtype of dementia. A consensus panel led by a consultant neurologist established the final diagnosis according to standard criteria for dementia (DSM-III-R) and Alzheimer’s disease (National Institute of Neurological and Communicative Disorders and Stroke–Alzheimer’s Disease and Related Disorders Association). All participants were followed for incident dementia until January 1, 2015. Follow-up was virtually complete (97.2% of potential person-years).
External Validation
For external validation of the models, we used the Epidemiological Prevention Study of Zoetermeer (EPOZ) from the Netherlands and the Alzheimer’s Disease Neuroimaging Initiative cohort 1 (ADNI-1) from the United States. EPOZ started in 1975, with the aim of assessing the prevalence of several chronic diseases and their determinants in the city of Zoetermeer (19). Response rates were similar to those of the Rotterdam Study (72%). In 1995 and 1996, a random subsample of the participants who were between ages 60 and 90 underwent cognitive testing and brain MRI (N=514); data from this group are considered the baseline for the present study. Participants were screened at study entry and at follow-up visits for dementia, using a strict protocol (20). All participants were followed for incident dementia until the end of study, on January 1, 2007 (follow-up included 90.8% of potential person-years). For validation within ADNI, we selected 228 cognitively unimpaired individuals age 60 or older. Data used in the preparation of this study were obtained from the ADNI database (adni.loni.usc.edu). The primary goal of ADNI has been to test whether serial MRI, PET, other biological markers, and clinical and neuropsychological assessments can be combined to measure the progression of mild cognitive impairment and early Alzheimer’s disease. Further details on ADNI can be found elsewhere (21).
Statistical Analysis
To reduce extreme effects of the predictors, we truncated the distribution of continuous variables at the 1st and 99th percentiles. Distributions for white matter hyperintensity volume and Stroop interference score were skewed. We obtained normal distributions of these parameters using a natural logarithmic transformation. We modeled potential nonlinear effects of age by using restricted cubic spline transformations and by adding an age-squared term, to capture most accurately the effects of age as the most important risk factor for dementia.
For the basic model, we used competing risk regression as proposed by Fine and Gray (22), with all candidate predictors included and fitted into the model to calculate 10-year risk of dementia. Table S2 in the online supplement provides further details on the development steps of the model and testing of the assumptions. We subsequently used the least absolute shrinkage and selection operator (LASSO) technique, adapted to a competing risk setting to simultaneously penalize the model’s regression coefficients and select important predictors for the final model (23, 24). The LASSO method is particularly useful for preventing model overfitting and model misspecification (25). An overfitted model tends to underestimate the probability of an event in low-risk groups and overestimate an event in high-risk groups.
For the development of the extended model, we used the predictors selected by the LASSO technique in the basic model as a starting point and extended it with the addition of objective cognitive tests, APOE-ε4 carrier status, and brain MRI parameters. As a reference, we used a model based on age alone for all analyses. In a stepwise exploratory analysis, we investigated the additive predictive value for each domain separately (cognition, imaging, and genetic information) of the final extended model, compared with the basic model. All of the C-statistics from the development sample presented here are corrected for optimism.
Internal Validation
We evaluated the robustness of the model using bootstrap samples for each model and found consistent results in selection steps and coefficient shrinkage using the LASSO technique, based on 200 bootstrap samples (see Tables S4 and S6 in the online supplement ) (26). We quantified the discriminative ability of these models using the C-statistic for survival data with competing outcomes (27, 28). The C-statistic is an adapted area-under-the-curve metric for use in survival analyses. It indicates the overall proportion of all pairs of participants that can be ordered such that the participant who developed dementia during follow-up indeed had a higher predicted risk. We used the cumulative incidence function to calculate the absolute 10-year risk of dementia (29). We used the DeLong test adapted for survival analyses to infer whether C-statistics of the basic and extended models were statistically different from those of a model based on age alone (30).
Stratified Analysis
We assessed the predictive accuracy for the most common subtype of dementia, namely, Alzheimer’s disease, and assessed model performance for men and women separately. Next, we excluded the first 4 years of follow-up to assess whether the predictive value extended beyond the first years of follow-up, since some of the predictors may reflect prodromal or undiagnosed dementia. To further investigate model robustness across varying time horizons, we evaluated the predictive ability of the model using 3-, 5-, and 15-year time horizons. Finally, we stratified on age (≤80 years and >80 years) at baseline, given the median age at diagnosis (31) and the steep increase in incidence of dementia beyond this age in order to investigate the performance of the model at different ages.
Results
Development Cohort Study Population
The baseline characteristics of the 2,710 participants of the development cohort are summarized in Table 1. The mean age was 71.2 years; 52.8% of the participants were women, and 33.3% had subjective memory decline. During a median follow-up of 7.0 years for those who were censored alive (interquartile range, 5.1–9.1), with a total follow-up of 20,324 person-years, 181 participants developed dementia, of whom 146 developed Alzheimer’s disease; 578 participants died from other causes. This corresponds to a crude incidence rate for dementia of 9.2 per 1,000 person-years. During the 10-year predicted time horizon, 131 participants developed dementia and 444 participants died free of dementia.
| Variable | Rotterdam Study (N=2,710) | EPOZ (N=514) | ADNI-1 (N=228) | |||
|---|---|---|---|---|---|---|
| Mean | SD | Mean | SD | Mean | SD | |
| Age (years) | 71.2 | 8.2 | 70.8 | 6.5 | 75.9 | 4.9 |
| Systolic blood pressure (mmHg) | 145 | 21 | 149 | 23 | 134 | 17 |
| Median | Range | Median | Range | Median | Range | |
| Education (years)b | 10 | 7–13 | 10 | 7–13 | 16 | 14–18 |
| N | % | N | % | N | % | |
| Female | 1,430 | 52.8 | 274 | 53.3 | 110 | 48.0 |
| Smoking | ||||||
| Ever | 1,884 | 69.5 | 326 | 63.4 | 85 | 37.3 |
| Current | 446 | 16.5 | 86 | 16.7 | ||
| History of diabetes | 345 | 12.7 | 38 | 7.4 | 18 | 7.9 |
| History of symptomatic stroke | 106 | 3.9 | 18 | 3.5 | 3 | 1.3 |
| Depressive symptoms | 457 | 16.9 | 39 | 7.6 | 34 | 14.9 |
| Parental history of dementia | 185 | 6.8 | 100 | 43.9 | ||
| Subjective memory decline | 903 | 33.3 | 177 | 34.4 | 17 | 7.5 |
| Need for assistance with finances or medication | 262 | 9.7 | 24 | 4.7 | 13 | 5.7 |
| APOE-ε4 carrier | 759 | 28.0 | 143 | 27.8 | 61 | 26.8 |
| Mean | SD | Mean | SD | Mean | SD | |
| Cognitive tests | ||||||
| Word fluency test, words | 21 | 5 | 21 | 5 | 20 | 5 |
| Letter digit substitution test, letters | 28 | 7 | 27 | 7 | 46 | 10 |
| Stroop interference task, seconds | 57 | 27 | 56 | 22 | — | |
| Delayed word learning test, words | 7 | 3 | 6 | 3 | 6 | 2 |
| Imaging markers | ||||||
| Total brain volume (mL) | 880.1 | 126.1 | 839.8 | 100.6 | 1,008c | 100.5 |
| Mean hippocampal volume (mL) | 3.7 | 0.5 | 2.7 | 0.4 | 3.6 | 0.4 |
| Median | Range | Median | Range | Median | Range | |
| White matter hyperintensity volume (mL)b | 4.7 | 0–143.6 | 1.5 | 0–25.6 | 0.24 | 0–25.5 |
| N | % | N | % | N | % | |
| Presence of infarcts | 410 | 15.1 | 92 | 17.9 | 18 | 7.9 |
a
Data presented are nonimputed data. Data were virtually complete for the Rotterdam Study (<7.4% missing), except for family history of dementia (19.2%) and need for assistance with finances or medication (23.8%). ADNI-1=Alzheimer’s Disease Neuroimaging Initiative cohort 1; APOE=apolipoprotein E; EPOZ=Epidemiological Prevention Study of Zoetermeer.
b
Median and range are presented because of skewed distributions.
c
Including cerebellar volumes.
Model Development and Internal Validation
There was evidence for a nonlinear relationship between age and risk of dementia. We therefore added age-squared into the model to capture this nonlinearity. The basic model considered age, age-squared, sex, education level, systolic blood pressure, current smoking, history of diabetes, history of symptomatic stroke, depressive symptoms, parental history of dementia, presence of subjective memory decline, and need for assistance with finances or medication. In Table 2, the subdistribution hazard ratios are presented.
| Basic Model | Extended Model | |||||||
|---|---|---|---|---|---|---|---|---|
| Original | LASSO | Original | LASSO | |||||
| Predictor | Hazard Ratio | 95% CI | Hazard Ratio | 95% CI | Hazard Ratio | 95% CI | Hazard Ratio | 95% CI |
| Age | 3.24 | 2.04, 5.14 | 1.09 | 1.07, 1.11 | 1.03 | 1.01, 1.06 | 1.03 | 1.01, 1.04 |
| Age-squared | 0.99 | 0.99, 1.00 | Not selected | |||||
| Female | 1.00 | 0.70, 1.43 | Not selected | |||||
| Education | 1.01 | 0.95, 1.06 | Not selected | |||||
| Systolic blood pressure, per 10 mmHg | 0.95 | 0.87, 1.03 | Not selected | |||||
| Current smoking | 0.85 | 0.50, 1.45 | Not selected | |||||
| History of diabetes | 1.26 | 0.80, 1.99 | Not selected | |||||
| History of symptomatic stroke | 2.29 | 1.32, 3.97 | 1.82 | 1.43, 2.22 | 1.26 | 0.68, 2.32 | 1.09 | 0.95, 1.22 |
| Depressive symptoms | 1.19 | 0.79, 1.78 | Not selected | |||||
| Parental history of dementia | 1.30 | 0.72, 2.35 | Not selected | |||||
| Subjective memory decline | 1.65 | 1.16, 2.35 | 1.31 | 1.13, 1.48 | 1.42 | 0.99, 2.04 | 1.18 | 1.03, 1.33 |
| Need for assistance with finances or medication | 1.80 | 1.17, 2.75 | 1.46 | 1.21, 1.72 | 1.38 | 0.88, 2.17 | 1.25 | 1.04, 1.45 |
| Word Fluency Test, words | 0.95 | 0.91, 0.99 | 0.96 | 0.94, 0.98 | ||||
| Letter digit substitution test, letters | 0.99 | 0.95, 1.02 | 0.99 | 0.98, 1.00 | ||||
| Stroop interference task, secondsb | 1.00 | 0.99, 1.01 | Not selected | |||||
| Delayed word learning test, words | 0.82 | 0.74, 0.90 | 0.84 | 0.78, 0.89 | ||||
| APOE-ε4 carrier | 1.91 | 1.31, 2.98 | 1.89 | 1.65, 3.41 | ||||
| Total brain volume, per 10% ICV | 0.37 | 0.19, 0.71 | 0.39 | 0.05, 0.74 | ||||
| Hippocampal volume, per 10% ICV | 0.46 | 0.31, 0.70 | 0.52 | 0.25, 0.80 | ||||
| Total white matter hyperintensity volume, % ICVb | 1.15 | 0.94, 1.41 | 1.10 | 1.02, 1.18 | ||||
| Infarcts (cortical/lacunar) | 0.92 | 0.59, 1.45 | Not selected | |||||
a
APOE=apolipoprotein E; ICV=intracranial volume; LASSO=least absolute shrinkage and selection operator.
b
Natural log transformed.
The discriminative accuracy measured with the C-statistic of the full basic model was 0.79 (95% CI=0.76, 0.83). After shrinkage and predictor selection using the LASSO technique, all statistically significant predictors remained in the model: age, history of symptomatic stroke, presence of subjective memory decline, and need for assistance with finances or medication. The C-statistic remained similar, at 0.79 (95% CI=0.76, 0.82). Based on 200 bootstrap samples, model optimism was small (that is, the predictive performance of the models was not tied to a specific sample; the optimism-corrected C-statistic for the basic model was 0.78 [95% CI=0.75, 0.81]; see Tables S4 and S6 in the online supplement ). From here, all presented C-statistics derived from the development study are corrected for optimism to represent optimism-corrected C-statistics.
Adding cognitive, APOE-ε4 carrier status, and imaging information to the basic model resulted in higher discriminative ability (C-statistic=0.86, 95% CI=0.83, 0.88). In Table S5 in the online supplement , C-statistics are presented when cognitive, APOE-ε4 carrier status, or imaging information are added to the basic model separately. After shrinkage and selection, the letter digit substitution test, the delayed word learning test, APOE-ε4 carrier status, and all imaging markers except brain infarcts were selected (C-statistic=0.86, 95% CI=0.83, 0.88). Ten-year risks based on the basic model are easily calculated using a simple risk chart (Figure 1). An Excel spreadsheet to calculate risks with the extended model is available as an online supplement .
FIGURE 1. Risk chart for calculating 10-year risk of dementia using the basic modela
a As an example, a 67-year-old man or woman without a history of stroke, with subjective memory complaints, and without difficulties managing finances or medication, has a 6% risk of developing dementia within 10 years.
Stratified Analyses
The basic and extended models showed roughly similar results for Alzheimer’s disease, and for men and women separately (Table 3). Discriminative ability remained similar when the first 4 years of follow-up were excluded (C-statistic=0.79, 95% CI=0.74, 0.84). Using 3- and 5-year predicted time horizons, the basic and extended models had higher discriminative properties (the C-statistics for the basic and extended models, respectively, were 0.82 [95% CI=0.78, 0.86] and 0.91 [95% CI=0.87, 0.95] for a 3-year horizon and 0.79 [95% CI=0.74, 0.83] and 0.88 [95% CI=0.85, 0.91] for a 5-year horizon), compared with the 10-year predicted time horizon (Table 4). In contrast, when using a 15-year predicted time horizon, the basic and extended models had lower discriminative properties (the C-statistics were 0.67 [95% CI=0.62, 0.74] and 0.71 [95% CI=0.65, 0.75], respectively). In individuals older than age 80 (N=456), the basic model showed considerably lower discriminative ability (C-statistic=0.57, 95% CI=0.49, 0.63). In contrast, the extended model retained substantial discriminative ability in this stratum (C-statistic=0.71, 95% CI=0.64, 0.76) and was considerably higher compared to a prediction based on age alone (C-statistic=0.53, 95% CI=0.45, 0.63).
| Age Alone | Basic Model | Extended Model | |||||
|---|---|---|---|---|---|---|---|
| Study | N/n | C-Statistic | 95% CI | C-Statistic | 95% CI | C-Statistic | 95% CI |
| Rotterdam Study | 2,710/131 | 0.76b | 0.73, 0.78 | 0.78b | 0.75, 0.81 | 0.86b | 0.83, 0.88 |
| EPOZ | 514/36 | 0.73 | 0.65, 0.82 | 0.75 | 0.67, 0.82 | 0.81 | 0.74, 0.88 |
| ADNI-1 | 228/26 | 0.54 | 0.42, 0.64 | 0.72c | 0.63, 0.83 | ||
a
ADNI-1=Alzheimer’s Disease Neuroimaging Initiative cohort 1; EPOZ=Epidemiological Prevention Study of Zoetermeer; N=number at risk; n=number of events.
b
Optimism-corrected C-statistics.
c
ADNI recruits participants via specialized clinical study sites, so we only tested the performance of the extended model.
| Age Alone | Basic Model | Extended Model | |||||
|---|---|---|---|---|---|---|---|
| Analysis | N/n | C-Statistic | 95% CI | C-Statistic | 95% CI | C-Statistic | 95% CI |
| Alzheimer’s disease only | 2,710/105 | 0.76 | 0.72, 0.80 | 0.77 | 0.75, 0.81 | 0.86 | 0.83, 0.88 |
| Men | 1,280/61 | 0.76 | 0.72, 0.82 | 0.79 | 0.75, 0.83 | 0.87 | 0.84, 0.92 |
| Women | 1,430/70 | 0.75 | 0.71, 0.79 | 0.78 | 0.73, 0.80 | 0.85 | 0.81, 0.88 |
| Excluding first 4 years of follow-up | 2,417/65 | 0.78 | 0.74, 0.81 | 0.79 | 0.76, 0.82 | 0.85 | 0.82, 0.89 |
| ≤80 years old | 2,254/79 | 0.77 | 0.74, 0.81 | 0.80 | 0.75, 0.84 | 0.87 | 0.84, 0.91 |
| >80 years old | 456/52 | 0.53 | 0.45, 0.63 | 0.57 | 0.49, 0.63 | 0.71 | 0.64, 0.76 |
| 3-year time horizon | 2,710/47 | 0.80 | 0.75, 0.84 | 0.82 | 0.78, 0.86 | 0.91 | 0.87, 0.95 |
| 5-year time horizon | 2,710/81 | 0.77 | 0.74, 0.80 | 0.79 | 0.74, 0.83 | 0.88 | 0.85, 0.91 |
| 15-year time horizonb | 523/81 | 0.62 | 0.57, 0.68 | 0.67 | 0.62, 0.74 | 0.71 | 0.65, 0.75 |
a
N=number at risk; n=number of events. C-statistics are corrected for optimism.
b
The 15-year predicted time horizon for all-cause dementia was used in a subsample of the study population with sufficient follow-up.
External Validation
The baseline characteristics of the Rotterdam Study and EPOZ participants were largely similar, whereas the ADNI-1 participants, on average, were older, attainted a higher education level, reported less memory decline, and were more likely to have a history of parental dementia (Table 1). During a median of 9.5 years (interquartile range, 7.6–11.4) of follow-up in EPOZ, 36 participants developed dementia and 120 participants died free of dementia. During a median follow-up time of 6.3 years (interquartile range, 2.0–8.0) in ADNI-1, 26 participants developed dementia. Within EPOZ, both the basic and the extended model showed discriminative performance in line with their performance in the development cohort (the C-statistics were 0.75 [95% CI=0.67, 0.82] for the basic model and 0.81 [95% CI=0.74, 0.88] for the extended model). The models were well calibrated (see Figure S2 in the online supplement ). As reference, a model based on age alone yielded a C-statistic of 0.73 (95% CI=0.65, 0.82) and resulted in significantly worse performance compared with the basic and extended models (p values <0.001 for both). Given that ADNI is not a population-based study and recruits participants through specialized clinical study sites, we only tested the performance of the full model. This yielded a lower C-statistic of 0.72 (95% CI=0.63, 0.83), reflecting a more homogeneous and older population, yet it also performed significantly better than a model based on age alone (C-statistic=0.54, 95% CI=0.42, 0.64, p=0.01).
Discussion
In this study, we developed and validated a simple prediction model for dementia in an aging population in primary care. In addition, we demonstrated that this performance can be further extended in a model including cognitive testing, APOE genotyping, and brain MRI parameters. These models can be used to calculate the 10-year risk of dementia to inform individuals and optimize risk stratification for clinical trials.
The discriminative ability of our basic model was comparable to previously published models incorporating data for use in primary care settings (9). Most previous studies only reported on discriminative ability, ranging from 0.65 to 0.80 as measured with the C-statistic. For instance, the Brief Dementia Screening Indicator, using data available in primary care, yielded C-statistics between 0.68 and 0.78 across four cohorts. Notably, four other prediction models included in a recent external validation study did not provide additional predictive value in dementia risk prediction compared with a model with age as the only predictor (10). In the present study, the basic model we developed did show greater discriminative ability and improved calibration above and beyond age alone. Compared with the Brief Dementia Screening Indicator model, our basic model additionally included the presence of subjective memory decline. The strength of this predictor in relation to the occurrence of dementia (adjusted hazard ratio=1.65) and the prevalence in the general population (33%) resulted in better predictive performance.
The models in this study include a history of stroke instead of the various individual cardiovascular risk factors included in several previous models (9). We did consider traditional cardiovascular risk factors, but they did not pass the mark for inclusion in the final models. Several explanations may underlie these observations. First, almost a quarter of all dementia cases can be attributed to vascular risk factors, illustrating their etiological importance in the development of dementia (18, 32, 33). However, similar to coronary heart disease prediction in the elderly (34, 35), the role of cardiovascular risk factors in dementia prediction may strongly diminish with age. Second, cardiovascular risk factors are also strongly associated with various other diseases in old age, reducing their specific discriminative ability in predicting the occurrence of dementia. For instance, smoking could lead to potentially fatal competing events, such as cardiovascular events or cancer at younger ages, and thereby preclude the occurrence of dementia. As a consequence, smoking has limited specificity in predicting cardiovascular disease, cancer, or dementia at older ages. Dementia risk prediction models should take into account competing risks to avoid uninterpretable C-statistics and inflated absolute risks (15). We dealt with this issue in this study by deriving our dementia prediction models within a competing risk framework.
In line with results from a previous model based on predictors derived in a primary care setting (36), our basic model had poor discriminative ability in individuals older than age 80. This finding is generally of limited concern when using a prediction model to identify high-risk individuals for clinical trials, since trials generally aim to recruit younger individuals. Yet, these findings provide insight into the complexity of dementia prediction using only clinical parameters in the oldest-old. In contrast, our extended model showed substantially higher discriminative ability for individuals older than age 80, highlighting the significance of objective markers of cognition and brain structure in the oldest-old, including cognitive testing, genetics, and brain imaging.
In this study, we developed and validated two complementary risk models: a basic model that could be used by family doctors and general practitioners, and an extended model that could be used in a specialized clinical care setting and that incorporates cognitive testing, brain MRI parameters, and genetic data. The strength of the extended model is that it uses information that reflects the underlying disease process. At the same time, it can be argued that the presence of these markers indicates that the disease is already present, raising the question whether the model offers prediction or in fact early diagnosis. Nevertheless, our sensitivity analyses excluding the first 4 years of follow-up showed similar predictive accuracies, suggesting that the effect of early diagnosis as opposed to prediction was marginal. Moreover, the ability to identify persons 10 years before clinical diagnosis can inform trials aimed at intervening in the earliest phase.
Indeed, it is now increasingly recognized that preventive or treatment strategies may be more effective when targeting individuals who are at increased risk of dementia (1, 6–9, 37). In order to direct such interventions to those who would most likely benefit from them, a reliable way to identify individuals at high risk for dementia is needed. The prediction models presented here address this gap, and they can be used to stratify individuals in clinical trials. Absolute 10-year dementia risk thresholds for determining low- and high-risk groups need to be established and may depend on the research question at hand, as well as the availability, costs, and risks of the intervention. These models can be combined in a two-step design, providing opportunities to identify at-risk individuals from the general population in primary care with a simple yet predictive model. Subsequently, these individuals can be referred to a more specialized care setting, where a more refined risk assessment can be done using the extended model. It would be interesting to investigate whether the performance of the basic model could be further improved with the addition of a simple blood test (38), or a brief cognitive test, such as the visual association test (39). The extended model could be further improved by adding novel imaging modalities, such as cerebral microbleeds or data on diffusor tensor imaging of the brain, by including rare genetic variants and functional genomics, or by extending the model with more in-depth neuropsychological tests (40–42). The predictive value of other predictors that were available either in the Rotterdam Study or in the validation studies could have been interesting to explore. However, in this study, we specifically aimed to develop a dementia prediction model and to validate exactly that model in these validation studies. Exploring the predictive yield of additional predictors would technically lead to the development or extension of another prediction model, which subsequently would have to be externally validated again.
The following limitations of this study must be considered. First, we used a regularization method (LASSO) that automatically selects and subsequently shrinks effect sizes of important predictors. This penalization strategy may have led to some underestimation of predictor effects in the development sample, but it increases the likelihood of replication in validation studies. Second, this study focused on older adults of predominantly Caucasian descent (>97%). Therefore, these models may not generalize to younger individuals or other ethnicities, and further validation work in these groups is needed. Third, we developed the models in a population-based setting, which matches the primary care setting, but this will likely affect model performance when validated or used in selected populations seen in specialized clinical care. This was in part reflected by a slightly lower discriminative accuracy in ADNI, yet in addition to differences in case mix (including the homogeneous character of this highly selected sample) and a relatively high attrition rate, discrimination remained substantially better than in a model based on age alone. Fourth, we used data on brain imaging with quantitative parameters, which may influence model performance compared with qualitative analyses, such as atrophy and white matter hyperintensity scales. Finally, dementia prediction without an effective therapy at hand raises ethical concerns. While such models are unlikely to be rolled out into clinical practice before further validation and assessment are undertaken, they have been shown to be useful for selecting individuals into clinical trials (2).
Strengths of the study include the large sample size and the availability of detailed information on a wide selection of potential dementia predictors. Moreover, the basic model is based on questionnaire information and is therefore simple to use, and it requires no further testing or laboratory measurements. Finally, the models were well validated, both internally and externally.
Conclusions
In this study, we developed and validated a dementia prediction model providing accurate dementia risk stratification and estimation in a general aging population. Addition of cognitive, imaging, and genetic features improved the predictive ability. These models can be used to identify individuals at high risk for dementia in the general population and may inform future clinical trial design.
Acknowledgments
The authors are grateful to the study participants, the staff from the Rotterdam Study, and the participating general practitioners and pharmacists. The authors acknowledge the support of Frank J.A. van Rooij as data manager.
The Rotterdam Study was approved by the Medical Ethics Committee of the Erasmus University Medical Center (registration number MEC 02.1015) and by the Dutch Ministry of Health, Welfare, and Sport (Population Screening Act, license number 1071272-159521-PG). The Rotterdam Study has been entered into the Netherlands National Trial Register (www.trialregister.nl) and into the WHO International Clinical Trials Registry Platform (www.who.int/ictrp/network/primary/en/) under the shared catalog number NTR6831.
The data used in preparation of this article were obtained from the ADNI database (adni.loni.usc.edu). The investigators within ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. A complete list of ADNI investigators can be found at http://adni.loni.usc.edu/wp-content/uploads/how_to_apply/ADNI_Acknowledgement_List.pdf.
Footnote
This work was supported by the project MULTIMODE, an EIT Health project; EIT Health is supported by EIT (European Institute of Innovation and Technology), a body of the European Union. The Rotterdam Study is sponsored by the Erasmus Medical Centre and Erasmus University Rotterdam, the Netherlands Organization for Scientific Research, the Netherlands Organization for Health Research and Development, the Research Institute for Diseases in the Elderly, the Netherlands Genomics Initiative, the Ministry of Education, Culture, and Science, the Ministry of Health, Welfare, and Sports, the European Commission (DG XII), and the Municipality of Rotterdam. Further support was obtained from the Netherlands Consortium for Healthy Aging and the Dutch Heart Foundation (2012T008). Data collection and sharing for this project were funded by the Alzheimer’s Disease Neuroimaging Initiative (ADNI) (NIH grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12-2-0012). ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, and through generous contributions from AbbVie, the Alzheimer’s Association, the Alzheimer’s Drug Discovery Foundation, Araclon Biotech, BioClinica, Biogen, Bristol-Myers Squibb, CereSpir, Cogstate, Eisai, Elan Pharmaceuticals, Eli Lilly, EuroImmun, Fujirebio, GE Healthcare, Hoffmann–La Roche and its affiliate Genentech, IXICO, Janssen Alzheimer Immunotherapy Research and Development, Johnson & Johnson Pharmaceutical Research and Development, Lumosity, Lundbeck, Merck, Meso Scale Diagnostics, NeuroRx Research, Neurotrack Technologies, Novartis Pharmaceuticals, Pfizer, Piramal Imaging, Servier, Takeda, and Transition Therapeutics. The Canadian Institutes of Health Research is providing funds to support ADNI clinical sites in Canada. Private sector contributions are facilitated by the Foundation for the National Institutes of Health (www.fnih.org). The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer’s Therapeutic Research Institute at the University of Southern California. ADNI data are disseminated by the Laboratory of Neuro Imaging at the University of Southern California.
Supplementary Material
File (appi.ajp.2018.18050566.ds001.pdf)
- View/Download
- 401.45 KB
File (appi.ajp.2018.18050566.ds002.xlsx)
- Download
- 13.51 KB
References
1.
Solomon A, Soininen H: Dementia: risk prediction models in dementia prevention. Nat Rev Neurol 2015; 11:375–377
2.
Ngandu T, Lehtisalo J, Solomon A, et al: A 2 year multidomain intervention of diet, exercise, cognitive training, and vascular risk monitoring versus control to prevent cognitive decline in at-risk elderly people (FINGER): a randomised controlled trial. Lancet 2015; 385:2255–2263
3.
Moll van Charante EP, Richard E, Eurelings LS, et al: Effectiveness of a 6-year multidomain vascular care intervention to prevent dementia (PreDIVA): a cluster-randomised controlled trial. Lancet 2016; 388:797–805
4.
Andrieu S, Guyonnet S, Coley N, et al: Effect of long-term omega 3 polyunsaturated fatty acid supplementation with or without multidomain intervention on cognitive function in elderly adults with memory complaints (MAPT): a randomised, placebo-controlled trial. Lancet Neurol 2017; 16:377–389
5.
Soininen H, Solomon A, Visser PJ, et al: 24-month intervention with a specific multinutrient in people with prodromal Alzheimer’s disease (LipiDiDiet): a randomised, double-blind, controlled trial. Lancet Neurol 2017; 16:965–975
6.
Kivipelto M, Mangialasche F, Ngandu T: Can lifestyle changes prevent cognitive impairment? Lancet Neurol 2017; 16:338–339
7.
Yassine HN: Targeting prodromal Alzheimer’s disease: too late for prevention? Lancet Neurol 2017; 16:946–947
8.
Sommerlad A, Livingston G: Preventing Alzheimer’s dementia. BMJ 2017; 359:j5667
9.
Tang EY, Harrison SL, Errington L, et al: Current developments in dementia risk prediction modelling: an updated systematic review. PLoS One 2015; 10:e0136181
10.
Licher S, Yilmaz P, Leening MJG, et al: External validation of four dementia prediction models for use in the general community-dwelling population: a comparative analysis from the Rotterdam Study. Eur J Epidemiol 2018; 33:645–655
11.
Verlinden VJA, van der Geest JN, de Bruijn RFAG, et al: Trajectories of decline in cognition and daily functioning in preclinical dementia. Alzheimers Dement 2016; 12:144–153
12.
Jessen F, Wiese B, Bachmann C, et al: Prediction of dementia by subjective memory impairment: effects of severity and temporal association with cognitive impairment. Arch Gen Psychiatry 2010; 67:414–422
13.
Stephan BC, Tzourio C, Auriacombe S, et al: Usefulness of data from magnetic resonance imaging to improve prediction of dementia: population based cohort study. BMJ 2015; 350:h2863
14.
Desikan RS, Fan CC, Wang Y, et al: Genetic assessment of age-associated Alzheimer disease risk: development and validation of a polygenic hazard score. PLoS Med 2017; 14:e1002258
15.
Koller MT, Raatz H, Steyerberg EW, et al: Competing risks and the clinical community: irrelevance or ignorance? Stat Med 2012; 31:1089–1097
16.
Ikram MA, Brusselle GGO, Murad SD, et al: The Rotterdam study: 2018 update on objectives, design, and main results. Eur J Epidemiol 2017; 32:807–850
17.
Ikram MA, van der Lugt A, Niessen WJ, et al: The Rotterdam Scan Study: design update 2016 and main findings. Eur J Epidemiol 2015; 30:1299–1315
18.
de Bruijn RF, Bos MJ, Portegies ML, et al: The potential for prevention of dementia across two decades: the prospective, population-based Rotterdam Study. BMC Med 2015; 13:132
19.
van Saase JL, Vandenbroucke JP, van Romunde LK, et al: Osteoarthritis and obesity in the general population: a relationship calling for an explanation. J Rheumatol 1988; 15:1152–1158
20.
Ikram MA, Vrooman HA, Vernooij MW, et al: Brain tissue volumes in relation to cognitive function and risk of dementia. Neurobiol Aging 2010; 31:378–386
21.
Mueller SG, Weiner MW, Thal LJ, et al: The Alzheimer’s Disease Neuroimaging Initiative. Neuroimaging Clin N Am 2005; 15:869–877
22.
Fine JP, Gray RJ: A proportional hazards model for the subdistribution of a competing risk. J Am Stat Assoc 1999; 94:496–509
23.
Tibshirani R: Regression shrinkage and selection via the LASSO. J R Stat Soc Series B Stat Methodol 1994; 58:267–288
24.
Fu Z, Parikh CR, Zhou B: Penalized variable selection in competing risks regression. Lifetime Data Anal 2017; 23:353–376
25.
Pavlou M, Ambler G, Seaman SR, et al: How to develop a more accurate risk prediction model when there are few events. BMJ 2015; 351:h3868
26.
Steyerberg EW, Bleeker SE, Moll HA, et al: Internal and external validation of predictive models: a simulation study of bias and precision in small samples. J Clin Epidemiol 2003; 56:441–447
27.
Wolbers M, Koller MT, Witteman JC, et al: Prognostic models with competing risks: methods and application to coronary risk prediction. Epidemiology 2009; 20:555–561
28.
Steyerberg EW: Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York, Springer, 2008
29.
Gray R: A class of K-sample tests for comparing the cumulative incidence of a competing risk. Ann Stat 1988; 16:1141–1154
30.
DeLong ER, DeLong DM, Clarke-Pearson DL: Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988; 44:837–845
31.
Wolters FJ, van der Lee SJ, Koudstaal PJ, et al: Parental family history of dementia in relation to subclinical brain disease and dementia risk. Neurology 2017; 88:1642–1649
32.
Norton S, Matthews FE, Barnes DE, et al: Potential for primary prevention of Alzheimer’s disease: an analysis of population-based data. Lancet Neurol 2014; 13:788–794
33.
Kivipelto M, Ngandu T, Laatikainen T, et al: Risk score for the prediction of dementia risk in 20 years among middle aged people: a longitudinal, population-based study. Lancet Neurol 2006; 5:735–741
34.
Koller MT, Leening MJ, Wolbers M, et al: Development and validation of a coronary risk prediction model for older US and European persons in the Cardiovascular Health Study and the Rotterdam Study. Ann Intern Med 2012; 157:389–397
35.
Krumholz HM, Seeman TE, Merrill SS, et al: Lack of association between cholesterol and coronary heart disease mortality and morbidity and all-cause mortality in persons older than 70 years. JAMA 1994; 272:1335–1340
36.
Walters K, Hardoon S, Petersen I, et al: Predicting dementia risk in primary care: development and validation of the Dementia Risk Score using routinely collected data. BMC Med 2016; 14:6
37.
Winblad B, Amouyel P, Andrieu S, et al: Defeating Alzheimer’s disease and other dementias: a priority for European science and society. Lancet Neurol 2016; 15:455–532
38.
Nakamura A, Kaneko N, Villemagne VL, et al: High performance plasma amyloid-β biomarkers for Alzheimer’s disease. Nature 2018; 554:249–254
39.
Jongstra S, van Gool WA, Moll van Charante EP, et al: Improving prediction of dementia in primary care. Ann Fam Med 2018; 16:206–210
40.
Mungas D, Jagust WJ, Reed BR, et al: MRI predictors of cognition in subcortical ischemic vascular disease and Alzheimer’s disease. Neurology 2001; 57:2229–2235
41.
Zeestraten EA, Lawrence AJ, Lambert C, et al: Change in multimodal MRI markers predicts dementia risk in cerebral small vessel disease. Neurology 2017; 89:1869–1876
42.
Staals J, Booth T, Morris Z, et al: Total MRI load of cerebral small vessel disease and cognitive ability in older people. Neurobiol Aging 2015; 36:2806–2811
Information & Authors
Information
Published In
History
Received: 16 May 2018
Revision received: 30 August 2018
Accepted: 17 September 2018
Published online: 11 December 2018
Published in print: July 01, 2019
Keywords
Authors
Competing Interests
Dr. Leening has received grants from Prins Bernhard Cultuurfonds, De Drie Lichten Foundation, Erasmus University Trustfonds, and the Albert Schweizer Hospital Research Fund with matching from the Oncology Research Albert Schweitzer Foundation and the Promoting Advanced Cardiology through Education Foundation; he has received personal fees from the American Heart Association, the Netherlands Epidemiology Society, the European Society of Cardiology, the Dutch Heart Foundation, and the Capri Cardiac Rehabilitation Foundation Rotterdam. The other authors report no financial relationships with commercial interests.
Funding Information
Research Institute for Diseases in the Elderly:
Ministerie van Onderwijs, Cultuur en Wetenschap10.13039/501100003245:
European Commission10.13039/501100000780:
Hartstichting10.13039/501100002996:
Netherlands Genomics Initiative
Erasmus Medisch Centrum10.13039/501100003061:
Erasmus Universiteit Rotterdam10.13039/501100001828:
Nederlandse Organisatie voor Wetenschappelijk Onderzoek10.13039/501100003246:
ZonMw10.13039/501100001826:
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).