Sensitivity and Specificity: DSM-IV Versus DSM-5 Criteria for Autism Spectrum Disorder

Initially DSM was developed for psychiatrists who were interested in describing and understanding the frequency with which mental illnesses develop in our society. In 1980, DSM-III moved from a descriptive or conceptual approach to an operationalized, criteria-defining approach to enable clinicians to make diagnoses on the basis of whether a patient's symptoms matched the diagnostic criteria. The expectation of DSM-III and the subsequent DSM-III-R and DSM-IV was that DSM-based research would identify the underlying etiologies of the disorders included in the manuals, which would allow greater refinement of the criteria and ultimately their validation by biological measures and etiologies. Now DSM-5 is being developed and most likely will be rolled out in 2013.

One of the major concerns of some mental health professionals and consumers is that the proposed DSM-5 new category of “autism spectrum disorder” (ASD) may exclude a substantial proportion of cognitively able individuals with pervasive developmental disorders (PDDs) other than autistic disorder, i.e., Asperger’s disorder or PDD not otherwise specified (PDD-NOS). This concern arose subsequent to some published reports. In the epidemiological study of Finnish children by Mattila et al. (1), which compared the sensitivity (i.e., ability to correctively identify those with the disease or disorder) of DSM-IV-TR and the first draft of the DSM-5 criteria for ASD, 12 (46%) of the 26 subjects with DSM-IV-TR PDDs (and full-scale IQs above 50) were identified as having ASD according to the DSM-5 criteria. In the study by McPartland et al. (2), when the first draft of the DSM-5 criteria for ASD were applied to a data set of 657 participants in a DSM-IV field trial evaluating clinical diagnoses of PDDs, 60.6% of those with a clinical diagnosis of a PDD would meet the DSM-5 criteria for ASD, indicating that 39.4% would not.

A commentary by the DSM-5 Neurodevelopmental Disorders Work Group (3) addressed serious methodological flaws in the study by McPartland and colleagues. The work group members considered that the archived data used in the analyses by McPartland et al. had too many inherent limitations for assessment of the criteria proposed for DSM-5, particularly in regard to sensitivity and specificity (i.e., ability to correctively identify those without the disease or disorder). The work group, however, also recognized that the group’s early analyses had limitations that would be addressed through the ongoing efforts to produce reliable, valid diagnostic criteria for DSM-5 that should be effective in identifying the broad array of individuals with ASD.

In this issue of the Journal, Huerta et al. (4) report a study designed to demonstrate the different rates of sensitivity and specificity based on the DSM-IV and DSM-5 ASD criteria. The study was also designed to provide supportive data to the DSM-5 ASD criteria. The study participants, ages 2 to 17, were obtained from three large data sets, resulting in 4,453 subjects with DSM-IV clinical diagnoses of (PDDs) (equivalent to ASD) and 690 subjects with non-PDD diagnoses (e.g., language disorder, attention deficit hyperactivity disorder). Items from a parent report measure of ASD symptoms (Autism Diagnostic Interview–Revised) and from a clinical observation instrument (Autism Diagnostic Observation Schedule) were matched to DSM-IV and DSM-5 criteria and then used to evaluate the sensitivity and specificity of the DSM-IV and DSM-5 criteria when compared with the clinical best-estimate diagnoses.

When only symptoms rated with the Autism Diagnostic Interview were used to identify children with ASD, the sensitivity of the DSM-5 criteria (0.91) was similar to that of the DSM-IV criteria for autistic disorder (0.91) in the combined study groups. The DSM-IV criteria for Asperger’s disorder and PDD-NOS had higher sensitivities (0.97 and 0.98, respectively). The results suggest that the DSM-IV criteria for the three subtypes of ASD had similar or better sensitivity than the DSM-5 criteria in terms of identifying ASD cases. On the other hand, the results seem to indicate that on the basis of the symptoms rated only with the Autism Diagnostic Interview, about 9% of children with clinical diagnoses of DSM-IV PDDs would not qualify for DSM-5 ASD.

Huerta and colleagues report that when only parent ratings from the Autism Diagnostic Interview were used to identify children with ASD in the total study group, the specificity of the DSM-5 criteria (0.53) was similar to the specificity of the DSM-IV criteria for autistic disorder (0.53), but the DSM-IV criteria for Asperger’s disorder and PDD-NOS had lower specificity (0.34 and 0.24, respectively). The overall accuracy of nonspectrum classification made by DSM-5 was slightly better than the accuracy of the DSM-IV criteria. Nevertheless, both the DSM-IV and DSM-5 criteria for ASD are moderate in correctly identifying children without ASD. The results support the clinical observation that even with a state-of-the-art research instrument such as the Autism Diagnostic Interview and the relatively well-developed DSM-IV, some borderline cases tend to be misdiagnosed as ASD because of the high sensitivity and moderate specificity of the DSM-IV criteria for ASD. This phenomenon may continue with the use of the DSM-5 criteria for ASD. Hence, there is much room for improvement with respect to specificity.

Some DSM-5 ASD supporters claim that patients who have DSM-IV PDDs but fail to qualify for DSM-5 ASD would not be left completely without mental health services because most of them would qualify for the new DSM-5 diagnosis of “social communication disorder.” It is a disappointment that Huerta and colleagues did not investigate this aspect.

Overall, the study by Huerta et al. does provide some information to further improve future versions of the DSM-5 criteria for ASD. Huerta et al. acknowledge limitations of their study: the composition of two of the data sets may not be fully representative of children typically referred for assessment of ASD; the study instruments are largely based on the DSM-IV criteria, which may not capture the behaviors that might fit into the DSM-5 criteria; and using archival data and symptom counts is not comparable to making clinical diagnoses. Huerta et al. also acknowledge that the results obtained from their study may not reflect the proposed DSM-5 ASD criteria’s true sensitivity and specificity. I completely agree with the thought that to assess the true sensitivity and specificity of the DSM-5 criteria for ASD requires real-time evaluation of all information gathered during the evaluation. However, the subjects in the study should include both children and adults. At the present, the true sensitivity and specificity of DSM-5 criteria for ASD are still unclear.

My major concern with the proposed DSM-5 definition and diagnostic criteria for ASD is the decision to consolidate the subtypes of DSM-IV PDDs within the overarching category of ASD. This change reflects the thought that the symptoms of these subtypes represent a continuum from mild to severe, rather than being distinct disorders. In that case, I believe the term “autism continuum disorder” would be more appropriate than “autism spectrum disorder.” On the other hand, in supporting the proposed DSM-5 ASD criteria, one of the key members of the work group, Dr. Francesca Happé (5), stated, “To date there is not a robust, replicated body of evidence to support the diagnostic distinction.” However, the literature review provided by Dr. Happé showed that about equal numbers of studies (about five studies each) reported “no difference” and a “significant difference” between autistic disorder and Asperger’s disorder according to the different variables examined. My own recent review of literature (6) showed that about three times as many studies showed a significant difference as showed no difference between the two disorders. Thus, the opponents of DSM-5 ASD can also claim that to date there is not a robust, replicated body of evidence to support the concept of an “autism continuum” (i.e., DSM-5 ASD).