Brain MR Radiomics to Differentiate Cognitive Disorders

Data used in the preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). The ADNI was launched in 2003 as a public–private partnership with the primary goal of testing whether serial MRI, positron emission tomography (PET), other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of mild cognitive impairment and early Alzheimer’s disease. We selected cases from the shared image collection ADNI-1, a 5-year study with a cohort of 200 cognitively normal individuals, 200 individuals with mild cognitive impairment, and 400 individuals with Alzheimer’s disease (44). The participants were divided into the assigned cognitively normal, mild cognitive impairment, and Alzheimer’s disease groups and underwent 3-T imaging at the following time points: baseline, 6, 12, 18 (mild cognitive impairment only), and 24 months. We categorized participants into three cognitive groups as assigned by ADNI-1: cognitively normal, mild cognitive impairment, and Alzheimer’s disease. Group specific inclusion criteria are available on ADNI’s website under the General Procedures Manual or under Study Design, Background and Rationale (45, 46). Briefly, cognitively normal participants have Mini-Mental State Exam (MMSE) scores between 24 and 30 (inclusive) and a CDR of 0, and are non-depressed, without mild cognitive impairment, and non-demented (45). Participants with mild cognitive impairment have MMSE scores between 24 and 30 (inclusive), a memory complaint, objective memory loss measured by education-adjusted scores on Wechsler Memory Scale Logical Memory II, a CDR of 0.5, absence of significant levels of impairment in other cognitive domains, essentially persevered activities of daily living, and an absence of dementia (45). Alzheimer’s disease participants have MMSE scores between 20 and 26 (inclusive), CDR of 0.5–2, abnormal memory function documented by scoring below the education-adjusted cutoff on the Logical Memory II subscale (Delayed Paragraph Recall) from the Wechsler Memory Scale, and meet the NINCDS/ADRDA criteria for probable Alzheimer’s disease (45).

The CDR score is obtained through semi-structured interviews with patients and informants to evaluate six domains: memory, orientation, judgment and problem solving, community affairs, home and hobbies, and personal care (47). Patients are then classified on the following ordinal scales: 0 (no impairment), 0.5 (questionable impairment), 1 (mild dementia), 2 (moderate dementia), or 3 (severe dementia). Typically, a score of 0.5 is given to individuals with a diagnosis of mild cognitive impairment (48, 49).

The initial participant selection criteria were as follows: available CDR score associated with the time of image acquisition and available 3-T T₁ scanning protocol to ensure maximum resolution for the image analysis.

We found 204 unique participants in ADNI-1 with available 3-T T₁ MR images. Image data were available for all participants at different time points ranging from baseline to month 24. Because we were interested in predicting static cognition levels (CDR scores, cognitive groups), the time point was irrelevant. We selected one time point per participant to ensure unique participants across groups. To maximize group sizes, we first selected participants with a CDR score of 2, who were in the minority. These participants were excluded from all the other groups. Next, participants with CDR scores of 1 and 0.5 were selected. All the remaining participants not assigned to any groups were placed in the CDR 0 group. Individuals with a CDR score of 3 were excluded due to our small sample size. Then, we proceeded to find the 3-T MR scan time points associated with the assigned group labels for participants. The image data acquired at the selected time points were used for analysis. Thirty-one participants in total were excluded. The exclusions were due either to a mismatch between imaging and CDR score acquisition date (N=21) or image unavailability (N=10). This led to a final sample size of 173 individuals: with 67 classified as non-impaired (CDR 0), 48 with questionable cognitive impairment (CDR 0.5), 39 with mild cognitive impairment (CDR 1), and 19 with moderate cognitive impairment (CDR 2).

Demographic and clinical characteristics of the included study participants are presented in Table 1 and Table 2. It is noteworthy that to receive a diagnosis of mild cognitive impairment or Alzheimer’s disease, in addition to clinician judgment, intra-individual decline must be obtained with serial cognitive measurements (multiple CDR scores over time) or by a history of change from previously attained levels (50). Thus, the numbers of participants between cognitive grouping and CDR scores differs.

MR images can have large intensity variations when acquired from different scanners or under different acquisition parameters. ADNI performs several preprocessing steps on magnetization-prepared rapid gradient-echo (MP-RAGE) sequence images. This includes gradwarp geometry distortion correction and B1 and N3 intensity non-uniformity corrections (51) to ensure comparability of images across devices and protocols. To ascertain the comparability of images across patients, we normalized all images to have a common mean and variance in CSF (52). Texture and volume analyses were performed using the normalized images.

The imaging data were imported into the MIPAV (Medical Image Processing, Analysis, and Visualization) application version 7.2.0 (53). To avoid resampling the images, we limited the segmentation of the hippocampus to the coronal view since it provided a common pixel spacing of (1.02, 1.02) mm across all patients. Experts identified three slices with the largest possible view of the bilateral hippocampi and manually placed rectangular regions of interest (ROIs) (16×16 pixels) on the area of the hippocampi, while avoiding inclusion of areas outside the hippocampus (Figure 1A) as much as possible. This segmentation process resulted in six ROIs (3 slices×2 hippocampi) per patient. This segmentation is considered greater than two dimensional and less than three dimensional (often referred to as 2.5D), and it improves the reliability of the sampling process. The ROIs were cropped out of the images and set aside for texture analysis. The individuals who manually placed ROIs on the hippocampi were blinded to the diagnosis; another blinded individual performed quality control checks to ensure ROIs were centrally placed.

Next, we acquired mean, standard deviation, and range of voxel intensities across the ROIs. (subsequently referred to as raw intensity features). We then mapped the dynamic ranges of intensities inside the ROIs to 0–255 as a preprocessing step for characterization of texture. Several statistical and spectral texture analysis methods are included in our radiomics pipeline. Textural features describing patterns or spatial distribution of voxel intensities were calculated from second-order statistical gray level co-occurrence matrices (GLCM) (54), Laplacian of Gaussian Histogram (LoGHist) (55), rotationally invariant Discrete Orthonormal Stockwell Transform (DOST) (56), Gabor filter banks (GFB) (57), and local binary patterns (LBP) (58). These methods were implemented in Python programming language using custom-written code and open-source libraries (59, 60). In total, we extracted 119 features per ROI: three raw intensity, 26 GLCM, 10 DOST, 36 LoGHist, 12 LBP, and 32 GFB features. Extensive details on these features can be found in Ranjbar et al., Patel et al., and Ramkumar et al. (42, 43, 61) To account for sampling variability, we averaged the features over slice without losing the laterality information, leading to a total of 238 texture features (119 per hippocampus) per patient.

We used the volbrain system for computation of hippocampal volumetric measurements. Given a stack of MR images, volBrain (62, 63) automatically segments parenchyma, brain tissues, macrostructure and subcortical structures (shown in Figure 1B) and reports volumetric measurements of the structures. For this study, we used two volumetric features for the hippocampus area including relative volume (%) and asymmetry index (%). Relative volume represents the sum of the hippocampi volumes in relation to the volume of the intracranial cavity. The asymmetry index is the difference between right and left volumes divided by their mean.

Age and sex differences between groups were tested using Student’s t-test and Pearson’s chi-square test, respectively. Statistical significance was defined as a p value <0.05. We performed univariate analysis to compare the difference in texture and volume feature values for both CDR groups and cognitive groups. The p values were adjusted for multiple comparisons using the Benjamini and Hochberg false discovery rate method (64).

We applied principal component analysis (PCA) to reduce dimensionality of texture features (65). To maintain interpretability of the principal components, PCA was applied to features stemming from a common texture analysis method. Several comparative datasets were generated with PCA to find the optimal level of variance. The final set of PCs represented 90% of the variance in the original features. Texture PCs combined with volume features were used in supervised classification of two label variables: cognitive groups (cognitively normal, mild cognitive impairment, Alzheimer’s disease) and CDR scores.

Machine learning was conducted using the open-source Python-based package scikit-learn (66) and custom-written scripts. We used a leave-one-out cross-validation (LOOCV) scheme to predict the labels (65) and to select features for training. LOOCV iteratively uses all samples except one for model training. In each round, the left-out sample serves as the test case to assess the generalizability of the trained model on an unseen case. In each round, a trained model was generated using features selected by Sequential Forward Feature Selection (SFFS) (65) and an internal cross-validation (CV). Starting from an empty set, SFFS sequentially added features as long as their addition resulted in CV accuracy improvement of 5%. We used diagonal quadratic discriminant analysis (DQDA) as the classification method (65). DQDA is a naïve Bayes classifier that allows for diagonal class covariance matrices and has shown to be successful in classification tasks of high-dimensional data with small sample sizes (67). Several studies have shown that DQDA has comparable or better performance than support vector machine in classification of high-dimensional data (68, 69).

Our data, by its nature, contained class imbalance, in which dominance of the majority class can hinder the classifier’s ability to learn the inherent properties of each class. To ensure generalizability of the result in experiments with substantial class imbalance, we used an ensemble down-sampling approach coupled with the above-mentioned learning scheme. In each CV round the training samples were divided into majority and minority groups. The majority group was then randomly divided into subsets roughly the same size as the minority group. Each of the subsets was merged with the minority group and served as the training set. The average probability across models for the test sample was used as the probability for that sample. This iterative process allowed every sample in the data set to serve as the left-out sample once.

The area under the receiver operating characteristic curve, sensitivity, and specificity were used to assess classification performance using the open-source software packages R (2.7) (70) and Scipy (0.15.1, Python 2.7) (71). The method of DeLong et al. and the pRoc package (72) were used to estimate the receiver operating characteristic (ROC) curve significance, p values, and 95% confidence intervals (73). The significance level (p<0.05) is the probability that the observed sample area under the ROC curve is significantly different from the null hypothesis (area=0.5) and is evidence that the model does have an ability to distinguish between the two groups.

The area under the ROC curves (AUCs) for the classification of cognitive groups is shown in Figure 4A. Classification reached AUC levels of 0.89 (CI=0.82–0.94) for cognitively normal compared with Alzheimer’s disease; 0.86 (CI=0.79–0.91) for cognitively normal compared with mild cognitive impairment; and 0.70 (CI=0.61–0.77) for mild cognitive impairment compared with Alzheimer’s disease. The performance measures, selected features, and ROC curve analysis for the cognitive groups are summarized in Table 3. All three models were significant at a p value ≤0.05. Including sex in the models did not affect the results.

The AUCs for the classification of CDR scores is shown in Figure 4B. The AUC levels of our models were: 0.98 (CI=0.93–0.99) for CDRs 0–2; 0.95(CI=0.9–0.98) for CDRs 0–1; 0.84 (CI=0.76–0.89) for CDRs 0–0.5; 0.73 (CI=0.61–0.83) for CDRs 0.5–2; 0.71 (CI=0.61–0.8) for CDRs 0.5–1; and 0.56 (CI=0.42–0.69) for CDRs 1–2. Overall, models were more successful in classification when the target groups were farther apart on the CDR spectrum. Details of the models’ performance and significance, selected features, and ROC curve statistics for this analysis are present in Table 4. All models were significant at a p value ≤0.05 except for the classification model CDR 1–2. Relative volume of hippocampi (percent volume) was a predictive feature in two of the six models. We conducted further analysis to assess whether age accounted for the significance of percent volume. When age was included in the model, percent volume remained highly statistically significant (p=0.003), while age was not significant (p=0.35). The AUC only slightly increased from 0.98 (model with percent volume alone) to 0.9910 (model with percent volume and age). A model containing age by itself resulted in an AUC of only 0.785, and the addition of percent volume significantly improved the model fit (p<0.0001). Thus, we conclude that percent volume is meaningful in differentiating between CDR 0 and 2, independent of age.