Screening for Cognitive Impairments After Traumatic Brain Injury: A Comparison of a Brief Computerized Battery With the Montreal Cognitive Assessment
Publication: The Journal of Neuropsychiatry and Clinical Neurosciences
Abstract
Detecting cognitive dysfunction in a busy traumatic brain injury (TBI) clinic is challenging given the length of conventional assessments and the need for psychometric expertise. The authors report the utility of a 10-minute, easily administered computerized battery that is more sensitive than the Montreal Cognitive Assessment in detecting cognitive impairments in people with a TBI.
Cognitive impairments are commonly experienced by persons with traumatic brain injury (TBI) and may contribute to difficulties performing instrumental and basic activities of daily living.1,2 In busy clinical settings, neuropsychological assessment can prove challenging due to the length of many cognitive batteries, the need for skilled psychometricians to administer tests, and, in certain health care settings, the cost. While brief screening tools come with fewer hurdles, they can at best give limited insights into a person’s cognition. One such commonly used instrument is the Montreal Cognitive Assessment (MoCA), previously utilized in a TBI setting.3,4
Given the importance of obtaining cognitive data in people with TBI, but cognizant of the limitations of an instrument such as the MoCA and the barriers to more comprehensive assessments, we have developed a brief computerized cognitive battery that takes less than 10 minutes to administer, requires little expertise, and gives immediate, automated results. The present study addresses the clinical usefulness of this approach in comparison to MoCA.
Methods
Subjects
Data were collected from a consecutive sample of 255 subjects (ages 14 to 62 years) attending a TBI clinic in a general hospital. Exclusion criteria included a history of another disease of the central nervous system, major psychiatric illness (dementia or psychosis), learning disability, current substance abuse, and a corrected visual acuity of less than 20/70.
Demographic and Injury-Related Variables
Demographic and TBI-related data included duration from time of injury to testing (days), level of education, past psychiatric history, durations of loss of consciousness and posttraumatic amnesia, and Glasgow Coma Scale scores, the latter obtained from the emergency room notes.
Cognitive and Psychosocial Assessment
All subjects were administered the MoCA and three computerized tests probing executive function, speed of information processing, and working memory. They included the Stroop Test, the Symbol Digit Modalities Test (SDMT), and a visual version of the Paced Auditory Serial Addition Test 2-second trials (PVSAT-2). Details of these computerized tests have been previously described and the tests validated against a comprehensive neuropsychological battery.5,6 A single research assistant trained in administering the computerized battery and MoCA tested all subjects. The research assistant was blind to other information about the study subjects. The order of testing was as follows: Stroop, SDMT, PVSAT-2, and MoCA. A total of 20 minutes was taken for cognitive assessment (10 minutes for the computerized battery, and 10 minutes for the MoCA).
Subjects were deemed to have failed a computerized test if results were more than 1.5 standard deviations below a mean normative score obtained from 200 healthy control subjects matched for age. For the MoCA, Z scores were calculated based on normative data controlling for age and level of education.7 For subjects with missing education information, MoCA normative corrections were adjusted only for age. Failure was then defined as a Z score 1.5 SD below the mean.
Subjects were also given two psychosocial questionnaires. The 28-item General Health Questionnaire (GHQ) detects psychological distress and consists of four subscales measuring somatic symptoms, anxiety, social dysfunction, and depression.8 A total score of 13 or greater is considered indicative of overall psychological distress in a neurological setting.8
The Rivermead Head Injury Follow-Up Questionnaire (RHIFUQ) measures psychosocial outcome in TBI subjects.9 It is a 10-item questionnaire reporting work, relationships, social, and domestic activities postinjury. Individual items are summed to give a total score.
Statistics
Cognitive data were checked for normality of distribution to establish the appropriateness of using parametric tests. Between-group (cognitively impaired vs. intact, or mild vs. moderate vs. severe TBI) comparisons were undertaken using two-sided t tests for continuous data and chi-square tests for categorical data. Correlations between the cognitive and behavioral (GHQ and RHIFUQ) variables were sought with a Pearson or Spearman correlation coefficient, depending on the nature of the data.
Ethics
Informed consent was obtained from all subjects, and the study was approved by the hospital’s research ethics board.
Results
Demographics Data
Injury-related and demographics description of the sample are shown in Table 1.
| Characteristic | Mean (SD)/Frequency (%) | |||
|---|---|---|---|---|
| Total Sample | Mild TBI | Moderate TBI | Severe TBI | |
| N | 255 | 198 | 36 | 21 |
| Age (years) | 36.20 (14.52) | 36.18 (14.35) | 36.11 (14.76) | 36.57 (16.33) |
| Gender (% male) | 152 (59.6%) | 113 (57.1%) | 27 (75.0%) | 12 (57.1%) |
| Disease severityb | ||||
| Mild | 198 (77.6%) | |||
| Moderate | 36 (14.1%) | |||
| Severe | 21 (8.2%) | |||
| Time between injury and assessment (days) | 96.41 (39.45) | 96.72 (39.68) | 97.28 (44.59) | 91.95 (27.43) |
| Level of educationc | ||||
| <12 years | 27 (10.6%) | 22 (11.1%) | 2 (5.6%) | 3 (14.3%) |
| =12 years | 97 (38.0%) | 67 (33.8%) | 20 (55.6%) | 10 (47.6%) |
| >12 years | 113 (44.3%) | 93 (47.0%) | 13 (46.1%) | 7 (33.3%) |
| Past psychiatric history (% yes) | 62 (24.3%) | 49 (24.7%) | 8 (22.2%) | 5 (23.8%) |
| Cognitive assessment | ||||
| Stroop Z score | –1.08 (2.05) | –1.12 (2.17) | –0.82 (1.48) | –1.18 (1.72) |
| SDMT Z score | –1.57 (2.08) | –1.52 (2.21) | –1.66 (1.64) | –1.91 (1.45) |
| PVSAT-2 Z score | –0.78 (1.22) | –0.77 (1.23) | –0.73 (1.15) | –1.09 (1.34) |
| MoCA Z score | 0.68 (0.77) | 0.72 (0.76) | 0.54 (0.76) | 0.58 (0.88) |
a
TBI, traumatic brain injury; SDMT, Symbol Digit Modalities Test; PVSAT-2, Paced Visual Serial Addition Test 2-second trials; MoCA, Montreal Cognitive Assessment.
b
TBI severity determined according to criteria defined by the Department of Defense (see reference 11).
c
N=237, with 18 subjects missing education information.
Cognitive and Psychosocial Data
The results of the cognitive assessment revealed that four subjects (1.6%) failed the MoCA based on Z score transformations. Of note is that this figure rises to 74 subjects (29%) had we just used the 26/30 MoCA impairment threshold without age and education correction. In those with a normal MoCA (N=251), 29.5% failed the Stroop, 37.5% the SDMT, and 24.3% the PVSAT-2. Overall, 54.98% of MoCA-intact subjects failed at least one test in the battery.
There were no significant differences in cognitive impairment frequencies between different TBI severity groups, as determined by categorical analyses using Pearson’s chi-square (χ2=1.98, df=2, p=.371 for Stroop; χ2=3.95, df=2, p=.139 for SDMT; and χ2=0.94, df=2, p=.625 for PVSAT-2). Given that only four subjects failed the MoCA, a similar chi-square analysis was not undertaken.
Associations Between Cognitive and Psychosocial Variables
In MoCA-intact subjects, significant differences in indices of psychological distress (GHQ) and psychosocial outcome (RHIFUQ) were found between subjects who passed and failed the Stroop and SDMT, with a trend evident in the PVSAT-2 (Table 2). A closer look at the GHQ subscale scores revealed that MoCA-intact subjects who failed the SDMT had greater somatic distress (t=3.051, df=235, p=0.003) and anxiety (t=2.186, df=235, p=0.030). Similarly, those who failed the Stroop had greater social difficulties (t=2.276, df=239, p=0.024) and depression (t=2.382, df=113.749, p=0.019). Concerning the RHIFUQ, those who failed the SDMT had more difficulties participating in simple and complex conversation (t=2.101, df=237, p=0.037, and t=–3.386, df=236, p=0.001, respectively) and enjoying leisure activities (t=1.986, df=236, p=0.048). Those who failed the PVSAT-2 had more difficulties in performing routine activities (t=–2.129, df=219, p=0.034). Finally, failure on the Stroop was associated with greater dysfunction in managing simple and complex conversation (t=2.862, df=117.721, p=0.005, and t=–2.924, df=240, p=0.004, respectively) and sustaining relationships with partners (t=−2.418, df=108.754, p=0.017).
| Comparison | ||||
|---|---|---|---|---|
| Stroop (N=241) | ||||
| Measure | Impaired mean (SD) | Not impaired mean (SD) | T | p |
| GHQ | 15.08 (8.30) | 12.10 (7.99) | –2.636 | 0.009 |
| RHIFUQ | 24.56 (13.63) | 21.15 (12.63) | –1.877 | 0.062 |
| SDMT (N=237) | ||||
| Measure | Impaired mean (SD) | Not impaired mean (SD) | T | p |
| GHQ | 14.80 (8.11) | 11.82 (8.05) | –2.764 | 0.006 |
| RHIFUQ | 24.53 (13.28) | 20.25 (12.74) | –2.487 | 0.014 |
| PVSAT–2 (N=219) | ||||
| Measure | Impaired mean (SD) | Not impaired mean (SD) | T | p |
| GHQ | 14.23 (8.60) | 12.34 (7.99) | –1.524 | 0.129 |
| RHIFUQ | 23.85 (13.31) | 21.41 (12.60) | –1.261 | 0.209 |
a
MoCA, Montreal Cognitive Assessment; GHQ, General Health Questionnaire; RHIFUQ, Rivermead Head Injury Follow-Up Questionnaire; SDMT, Symbol Digit Modalities Test; PVSAT-2, Paced Visual Serial Addition Test 2-second trials.
To address a possible association between depression and anxiety on the one hand and cognition and psychosocial outcome on the other, correlations were sought between GHQ depression/anxiety subscale scores and performance on the computerized battery and RHIFUQ. This revealed weak, albeit statistically significant, correlations between the GHQ depression subscale scores and performance on the Stroop (r=0.197, p=0.02) and SDMT (r=0.142, p=0.027). Weak correlations were also found between the GHQ anxiety subscale scores and performance on the Stroop (r=0.130, p=0.042) and SDMT (r=0.158, p=0.014). Last, there were stronger correlations between the GHQ depression subscale scores and the RHIFUQ (r=0.552, p<0.001), and between the GHQ anxiety subscale scores and the RHIFUQ (r=0.431, p<0.001).
An association was then sought between the presence of a premorbid psychiatric history and cognitive measures. This revealed that a premorbid psychiatric history including depression and anxiety was not significantly correlated with psychometric scores on cognitive tests (r=0.058, p=0.367 for Stroop, r=0.020, p=0.755 for SDMT, r=0.069, p=0.305 for PVSAT-2, and r=0.018, p=0.785 for MoCA). Past psychiatric history was also not significantly correlated with total score on the GHQ (rs=0.109, p=0.093), although it was weakly correlated with total score on the RHIFUQ (rs=0.143, p=0.026).
There was an association between total scores on the GHQ and RHIFUQ for the entire sample (r=0.686, p<0.001).
Discussion
Our data reveal that the MoCA lacks sensitivity in detecting cognitive impairment in TBI subjects. In contrast, our brief computerized cognitive assessment is able to detect impairment in more than 50% of the subjects deemed cognitively intact on the MoCA. Furthermore, the deficits elicited are independent of premorbid psychiatric history and come with real-world implications, namely greater psychosocial dysfunction. Before discussing these results in more detail, a brief comment on our sample composition is required. The percentage breakdown in our sample into mild, moderate, and severe TBI approximates the distribution of TBI severity in general.10 Moreover, the fact that we enrolled a consecutive sample of attendees from our clinic suggests that our sample composition is broadly representative of TBI subjects in general.
The MoCA has been used previously in a TBI setting and reportedly detected cognitive impairment in 88.3% of the sample.3 This far exceeds the percentage impairment in our study, a difference attributable to the utilization of a single-point cutoff score of 26 and to time of testing, namely three weeks postinjury (as opposed to our three months). Notably, our sample yielded a failure rate of 29.0% on the MoCA using the 26/30 cutoff score, which essentially disappeared after Z score transformation. Thus, our study exposes an important limitation of interpreting the MoCA with a single-point cutoff score, namely the possibility of overestimating the level of cognitive impairment. As such, cognitive impairment rates reported in previous uncorrected MoCA studies using the 26/30 cutoff should be viewed with caution.
Cognitive dysfunction in TBI is associated with many varied indices of poor real-world outcome. It has been linked to greater difficulties in finding and/or sustaining employment,12 maintaining relationships,13 pursuing leisure activities,13 and achieving a good quality of life.14 Our computerized data support this, given their association with scores on the RHIFUQ, a composite measure of psychosocial outcome. Furthermore, these real-world indices are linked to emotional well-being,15 which our data underscore; for we have shown that subjects who failed the Stroop and SDMT had a level of distress that exceeds the threshold score for psychiatric “caseness” according to the 28-item GHQ. Thus, the preliminary findings emerging from our computerized battery tell a coherent story in people with a TBI, namely that cognitive failings are associated with more challenges at work and in relationships and a higher degree of emotional distress.
Having demonstrated that the brief computerized battery is more sensitive than the MoCA in detecting cognitive compromise in this sample, it is important to interpret this result within the context of certain methodological limitations. First, the MoCA and our computerized battery differ somewhat in the scope of what they measure. The MoCA is a single composite score derived from many different cognitive domains, whereas each computerized test is more selective in its focus. That said, the three tests do capture an array of cognitive failings known to occur frequently in people with a TBI, namely impaired information processing speed (SDMT, PVSAT), working memory (PVSAT), and executive function (Stroop).1 Thus, taken collectively, these tests measure multiple aspects of cognition, as does the MoCA, albeit with a different emphasis resulting in a markedly improved sensitivity.
Second, it should be emphasized that associations do not imply causality. While it is parsimonious to attribute the cognitive failings on the computerized battery to the effects of cerebral trauma, other factors may also be playing a role. While our data ruled out premorbid psychiatric factors, such as depression and anxiety, emotional distress at the time of testing may contribute to impaired performance on the cognitive tests, as our data demonstrated. Similarly, these factors may contribute to greater self-reported functional difficulties, as measured by the RHIFUQ. Additional factors, such as pain, headache, and dizziness, to mention but a few, may also be important, but our methodology was not geared toward teasing out these potential influences.
Third, our computerized battery did not differentiate between mild, moderate, and severe TBI subjects. Here the possibility of type II error should be considered, given the small sample sizes of the moderate and severe groups relative to the mild TBI group.
Within the context of these limitations raised above, our computerized battery offers some important advantages. It is brief (10 minutes or less), generates automated, immediate results for the Stroop and SDMT, requires no special training, and is not dependent on motor involvement, a plus when assessing TBI subjects with upper limb injuries. Moreover, serial versions are available for longitudinal testing. Finally, and most important, the battery fits easily into a busy outpatient TBI clinic. While our computerized screening approach should not be misconstrued as a replacement for detailed cognitive inquiry, it is more sensitive than the MoCA in detecting cognitive compromise and provides clinically useful, rapid results in the absence of a neuropsychological service.
Acknowledgments
The authors thank Sofiya Sivalingam (Sunnybrook Health Sciences Centre) and Elke McLellan, OT (Sunnybrook Health Sciences Centre), for assisting with subject recruitment, test administration, and data collection.
References
1.
Dikmen SS, Corrigan JD, Levin HS, et al: Cognitive outcome following traumatic brain injury. J Head Trauma Rehabil 2009; 24:430–438
2.
Langlois JA, Rutland-Brown W, Wald MM: The epidemiology and impact of traumatic brain injury: a brief overview. J Head Trauma Rehabil 2006; 21:375–378
3.
de Guise E, Leblanc J, Champoux MC, et al: The Mini-Mental State Examination and the Montreal Cognitive Assessment after traumatic brain injury: an early predictive study. Brain Inj 2013; 27:1428–1434
4.
de Guise E, Alturki AY, LeBlanc J, et al: The Montreal Cognitive Assessment in persons with traumatic brain injury. Appl Neuropsychol Adult 2014; 21:128–135
5.
Lapshin H, Audet B, Feinstein A: Detecting cognitive dysfunction in a busy multiple sclerosis clinical setting: a computer generated approach. Eur J Neurol 2014; 21:281–286
6.
Lapshin H, Lanctôt KL, O’Connor P, et al: Assessing the validity of a computer-generated cognitive screening instrument for patients with multiple sclerosis. Mult Scler 2013; 19:1905–1912
7.
Rossetti HC, Lacritz LH, Cullum CM, et al: Normative data for the Montreal Cognitive Assessment (MoCA) in a population-based sample. Neurology 2011; 77:1272–1275
8.
Bridges KW, Goldberg DP: The validation of the GHQ-28 and the use of the MMSE in neurological in-patients. Br J Psychiatry 1986; 148:548–553
9.
Crawford S, Wenden FJ, Wade DT: The Rivermead Head Injury Follow Up Questionnaire: a study of a new rating scale and other measures to evaluate outcome after head injury. J Neurol Neurosurg Psychiatry 1996; 60:510–514
10.
Institute of Medicine. Gulf War and Health, vol. 7: Long-Term Consequences of Traumatic Brain Injury. Washington, DC, National Academies Press, 2009
11.
Department of Defense: VA/DoD Clinical Practice Guideline for Management of Concussion/Mild Traumatic Brain Injury (mTBI), Version 1.0. Washington, DC, Department of Veterans Affairs, Department of Defense, 2009
12.
Ponsford JL, Spitz G: Stability of employment over the first 3 years following traumatic brain injury. J Head Trauma Rehabil 2015; 30:E1–E11
13.
Ponsford JL, Downing MG, Olver J, et al: Longitudinal follow-up of patients with traumatic brain injury: outcome at two, five, and ten years post-injury. J Neurotrauma 2014; 31:64–77
14.
Dijkers MP: Quality of life after traumatic brain injury: a review of research approaches and findings. Arch Phys Med Rehabil 2004; 85(Suppl 2):S21–S35
15.
Draper K, Ponsford J, Schönberger M: Psychosocial and emotional outcomes 10 years following traumatic brain injury. J Head Trauma Rehabil 2007; 22:278–287
Information & Authors
Information
Published In
History
Received: 10 January 2016
Revision received: 27 February 2016
Accepted: 13 March 2016
Published online: 3 June 2016
Published in print: Fall 2016
Authors
Funding Information
The authors report no financial relationships with commercial interests.
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBGet Access
Login options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).