Can Computerized Cognitive Tests Assist in the Clinical Diagnosis of Attention-Deficit Hyperactivity Disorder?
Publication: The Journal of Neuropsychiatry and Clinical Neurosciences
Abstract
A group of 34 children and adolescents suspected of having attention-deficit hyperactivity disorder were referred for a computerized evaluation that included sustained attention, working memory, planning, and set-shifting. Although only sustained attention had reasonable specificity, all tests had questionable contribution to the diagnostic evaluation.
The assessment and diagnosis of attention-deficit hyperactivity disorder (ADHD) is clinical. It is based primarily on behavioral symptoms gathered via clinical examination, patient history, and reports and questionnaires. In the clinical realm, the diagnostic procedure is often a complex one for a variety of reasons: difficulty in obtaining valid information, additional clinical conditions that present with similar symptoms, comorbid conditions, and patients who meet borderline criteria.1,2 Since cognitive difficulties are known to be associated with behavioral symptoms, it is not surprising that professionals in the field are interested in finding objective tools to further validate the diagnosis.
Findings of impaired sustained attention functions were frequently reported1,3 and are therefore considered by some to be central to the understanding of the symptoms of ADHD.4 Thus, continuous performance tests (CPT) that examine these functions are probably the most commonly-used computerized neurocognitive assessments aiding in the clinical diagnosis of ADHD. The Test of Variables of Attention (TOVA) is a commonly-used CPT, with an estimated sensitivity of approximately 85% and specificity of approximately 70%.4 Because of its lower specificity, it carries a risk of overdiagnosis and treatment of children who do not have ADHD.1 One of the important issues related to assessing the sensitivity, specificity, and predictive value of a test is the population selected and the prevalence of the disorder in that population. Most studies used a healthy-control group2,5 or a well-established other diagnosis such as a learning disorder or conduct disorder.2,4 Whereas the study of core deficits in ADHD is based on differentiation from healthy volunteers, referrals differ from healthy volunteers in the sense that they are referred because of some difficulty. Thus, in clinical practice, the validity of the tests should be evaluated in relation to referrals and not to the asymptomatic population or another specific diagnosis. In a previous study,5 the CPT failed to differentiate between children with ADHD and children with subclinical levels of behavioral or cognitive problems, after excluding other diagnoses.
Some consider executive functions (EFs) as the core deficit in ADHD. Among the heterogeneous EFs, working memory, set-shift, response inhibition, and planning were frequently reported to be impaired in patients with ADHD.4,6–9 Most of the studies that relate to difficulties in these EFs used a normal-control group. The use of various neurocognitive batteries examining a range of EFs is becoming more popular in clinical practice.10
Still, despite the well-replicated EF weakness in ADHD subjects, variability between studies has been noted.8 Although the literature relates to other possible core findings, including delay-aversion, low frustration-tolerance, and emotional lability, neurocognitive batteries assessing CPT and EF have gained popularity in clinical practice.
The aim of the current study is to estimate the potential benefit of commonly used EF and sustained-attention assessment in the establishment of the diagnosis of ADHD, in a clinical setting of ADHD.
METHOD
A group of 34 children and adolescents, 19 boys and 15 girls, between the ages of 7 and 17, participated in the study (mean age: 11.5 years). Study protocol and consent forms were approved by the Institutional Review Board. All of the participants were children who had been referred by a neurologist or a child psychiatrist for a neurocognitive evaluation in order to substantiate a possible diagnosis of ADHD. Criteria for exclusion were known diagnosis of mental retardation or major psychopathology (namely, major affective disorder, psychotic disorder, pervasive developmental disorder, substance abuse, posttraumatic stress disorder, obsessive-compulsive disorder, panic disorder). After parents signed the informed consent form (and minors signed an assent form), all patients underwent the following:
1. Questionnaires: The Swanson, Nolan, & Pelham Questionnaire (SNAP–IV), a well-validated DSM–IV based questionnaire,11 was completed by one of the parents and, on a separate form, by a teacher; The Strengths and Difficulties Questionnaire (SDQ), a widely-used dimensional measure of child mental health,12,13 was completed by parents and children.
2. Structured interview: MP, a research psychologist trained in ADHD evaluation, performed a clinical interview separately for both child and parents according to the Development and Well-Being Assessment (DAWBA). The DAWBA is constructed to generate ICD–10 and DSM–IV psychiatric diagnosis for minors.14
3. Clinical interview: YB, a certified child and adolescent psychiatrist, blinded to the result of the structured interview, performed a clinical interview of child and parents.
The clinical diagnosis of ADHD was based on consensus between the research team, blinded to results of the computerized test.
4. Neurocognitive tests: For testing sustained attention, we administered the TOVA.1,5 Subtests of the CANTAB were used to evaluated the following EF domains: Working Memory, Spatial Working Memory (SWM); Planning – Stocking of Cambridge (SOC), and Cognitive Set-Shifting – Intradimensional/Extradimensional Shift (IED).4
Statistical Analysis
All variables were defined dichotomously. The clinical diagnostic was the “gold standard” to differentiate between ADHD and non-ADHD patients. For each cognitive domain, we used one standard deviation (SD) as cutoff for the following measures: TOVA: “ADHD score;” CANTAB SWM – between errors or strategy; CANTAB SOC problems solved in minimum moves or mean initial (and/or) subsequent thinking time; CANTAB IED stage completed or total errors. The association between the clinical evaluation and Cognitive tasks (TOVA and CANTAB) was tested by chi-square test or Fisher's exact test (when the expected count was <5 in at least one cell). Significant association was considered when α level was <0.05. Effect size was computed using the Pearson phi coefficient (rφ). For comparison of the efficiency of the different tests, we calculated the following measures: sensitivity: CPT/EF identified clinical ADHD from all ADHD (clinical) cases; specificity: CPT/EF identified non-clinical ADHD as not ADHD from all non-ADHD (clinical) cases; positive predictive value; CPT/EF identified clinical ADHD from total CPT/EF identified ADHD and negative predictive value; CPT/EF identified non-clinical ADHD as not ADHD from all non-ADHD CPT/EF-identified cases.
RESULTS
In the group, 27 of the 34 patients recruited were diagnosed clinically as suffering from ADHD. For 7 patients, the diagnosis of ADHD was excluded. These patients were subsequently diagnosed, two with dysthymia, two with conduct disorder and learning disabilities, two with anxiety disorder, and one with pervasive developmental disorder.
There was a significant association between the final clinical diagnosis and the findings of the CPT test (TOVA; p <0.05; Fisher's exact test (FET); rφ=0.39). Of the 27 patients with a confirmed diagnosis of ADHD, the TOVA verified attention-deficit in 17 and yielded non-pathological findings in 10 patients; thus, the TOVA had a sensitivity of 63%.
Of the seven patients who were diagnosed with another disorder (not ADHD), six had a non-pathological TOVA result; thus, the TOVA had a specificity of 85%. Since 17 of the 18 patients with the pathological result in the TOVA test had a confirmed diagnosis of ADHD, the positive predictive value was 94%. Since, of 16 non-pathological TOVA tests, only 6 were confirmed as not having ADHD, the negative predictive value was 37%.
The tests for executive functions (EFs) were not found to be significantly associated with the clinical diagnosis of ADHD (IED, SOC, SWM; p: NS, FET). Both the sensitivity (IED: 71%; SOC: 71%; SWM: 57%) and the specificity (IED: 7%; SOC: 11%; SWM: 22%) of the tests chosen from the CANTAB were low.
DISCUSSION
This study relates to the common clinical dilemma of diagnosing children with difficulties who are suspected to be suffering from ADHD. The study evaluated the contribution of computerized cognitive tests in the diagnosis of children with ADHD.
The central finding of the study revealed that both the EF tests examined and the CPT did not have a clinically significant contribution to the diagnosis. Although the CPT theoretically had a reasonable specificity of 85%, the fact that approximately 80% of the presented cohort suffered from ADHD diminishes its contribution.
This finding is opposed to the robust findings of malfunctions in these tests in previous research when comparing patients with ADHD to matched-control subjects.8,15 In our view, the reasonable explanation for this discrepancy lies in the choice of the control group. Although patients with ADHD have measurable deficits in functions related to EF as compared with non- clinical controls, these differences do not typify them in comparison to children who experience other disorders, such as anxiety or depression. Thus, our findings join previous studies that did not find a diagnostic contribution of cognitive batteries in differentiating ADHD patients from children with subclinical difficulties.5
The presented study has several limitations: a small sample size and high variability in the patients' ages. Larger studies with more homogenous groups might provide more detailed information about the exact nature of the impairments found by computerized cognitive tests in true clinical samples.
Acknowledgments
We thank Eve Leibowitz for her editorial assistance during the preparation of the manuscript.
None of the authors has any biomedical financial interests or potential conflicts of interest whatsoever in the publication of this research.
References
1.
Schatz AM, Ballantyne AO, Trauner DA: Sensitivity and specificity of a computerized test of attention in the diagnosis of attention-deficit/hyperactivity disorder. Assessment 2001; 8:357–365
2.
Sergeant JA, Geurts H, Oosterlaan J: How specific is a deficit of executive functioning for attention-deficit/hyperactivity disorder? Behav Brain Res 2002; 130(1,2):3–28
3.
Forbes GB: Clinical utility of the test of variables of attention (TOVA) in the diagnosis of attention-deficit/hyperactivity disorder. J Clin Psychol 1998; 54:461–476
4.
Willcutt EG, Doyle AE, Nigg JT, et al.: Validity of the executive function theory of attention-deficit/hyperactivity disorder: a meta-analytic review. Biol Psychiatry 2005; 57:1336–1346
5.
Preston AS, Fennell EB, Bussing R: Utility of a CPT in diagnosing ADHD among a representative sample of high-risk children: a cautionary study. Child Neuropsychol 2005; 11:459–469
6.
Barnett R, Maruff P, Vance A, et al.: Abnormal executive function in attention-deficit hyperactivity disorder: the effect of stimulant medication and age on spatial working memory. Psychol Med 2001; 31:1107–1115
7.
Goldberg MC, Mostofsky SH, Cutting LE, et al.: Subtle executive impairment in children with autism and children with ADHD. J Autism Dev Disord 2005; 35:279–293
8.
Doyle AE: Executive functions in attention-deficit/hyperactivity disorder. J Clin Psychiatry 2006; 67:21–26
9.
Toplak ME, Bucciarelli SM, Jain U, et al.: Executive functions: performance-based measures and the Behavior Rating Inventory of Executive Function (BRIEF) in adolescents with attention-deficit/hyperactivity disorder (ADHD). Child Neuropsychol 2009; 15:53–72
10.
Nichols SL, Waschbusch DA: A review of the validity of laboratory cognitive tasks used to assess symptoms of ADHD. Child Psychiatry Hum Dev 2004; 34:297–315
11.
Swanson JM, Kraemer HC, Hinshaw SP, et al.: Clinical relevance of the primary findings of the MTA: success rates based on severity of ADHD and ODD symptoms at the end of treatment. J Am Acad Child Adolesc Psychiatry 2001; 40:168–179
12.
Mansbach-Kleinfeld I, Apter A, Farbstein I, et al.: A population-based psychometric validation study of the Strengths and Difficulties Questionnaire, Hebrew version. Front Psychiatry 2010; 1:151
13.
Goodman R: Psychometric properties of the Strengths and Difficulties Questionnaire. J Am Acad Child Adolesc Psychiatry 2001; 40:1337–1345
14.
Goodman R, Ford T, Richards H, et al.: The Development and Well-Being Assessment: description and initial validation of an integrated assessment of child and adolescent psychopathology. J Child Psychol Psychiatry 2000; 41:645–655
15.
Pineda DA, Puerta IC, Aguirre DC, et al.: The role of neuropsychologic tests in the diagnosis of attention-deficit hyperactivity disorder. Pediatr Neurol 2007; 36:373–381
Information & Authors
Information
Published In
History
Received: 12 January 2011
Revision requested: 30 May 2011
Accepted: 9 June 2011
Published online: 1 January 2012
Published in print: Winter 2012
Keywords
Authors
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).